Max level shown:
1 starting November 7, 2024 at 02:03:11 PM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.34780905
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.06595444
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.061399166
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start ds: 8820
14 start ds: 8830
15 start access at:127.0.0.1:9998
16 Starting downstairs at port 8820
17 Starting downstairs at port 8810
18 Make output file at "/tmp/dsc/downstairs-8810.txt"
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Starting downstairs at port 8830
21 Make output file at "/tmp/dsc/downstairs-8830.txt"
22 [8820][1] initial start wait reports Starting
23 [8810][0] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1189)
26 [8810][0] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1190)
28 [8820][1] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1191)
30 [8830][2] initial start wait reports Running
31 All downstairs are running
32 Nov 07 14:03:11.869 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Nov 07 14:03:16.970 INFO accepted connection, remote_addr: 127.0.0.1:38046, local_addr: 127.0.0.1:9998
36 Nov 07 14:03:16.972 INFO request completed, latency_us: 474, response_code: 200, uri: /regioninfo, method: GET, req_id: d853570d-9ce5-42b6-83a6-48a7d50e22ed, remote_addr: 127.0.0.1:38046, local_addr: 127.0.0.1:9998
37 Nov 07 14:03:16.975 INFO request completed, latency_us: 108, response_code: 200, uri: /regioncount, method: GET, req_id: 9a82e4cb-4142-4932-966c-93a26554db64, remote_addr: 127.0.0.1:38046, local_addr: 127.0.0.1:9998
382024-11-07T14:03:16.974ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392024-11-07T14:03:16.975ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Nov 07 14:03:16.976 INFO request completed, latency_us: 171, response_code: 200, uri: /port/cid/0, method: GET, req_id: 1d56f058-de3d-468e-bb2a-0b30dd1ee696, remote_addr: 127.0.0.1:38046, local_addr: 127.0.0.1:9998
41 Nov 07 14:03:16.976 INFO request completed, latency_us: 101, response_code: 200, uri: /port/cid/1, method: GET, req_id: 45f79562-faa7-418b-b224-cdbb8300517c, remote_addr: 127.0.0.1:38046, local_addr: 127.0.0.1:9998
42 Nov 07 14:03:16.977 INFO request completed, latency_us: 94, response_code: 200, uri: /port/cid/2, method: GET, req_id: cdb95070-0169-4b2c-9722-6fea41ad2baa, remote_addr: 127.0.0.1:38046, local_addr: 127.0.0.1:9998
432024-11-07T14:03:16.977ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442024-11-07T14:03:16.978ZINFOcrucible: Upstairs starts
452024-11-07T14:03:16.978ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "311c3ca2c97d568407ae06780269d96fcfdd3a7c", git_commit_timestamp: "2024-11-07T13:47:04.000000000Z", git_branch: "renovate/tokio-1.x-lockfile", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462024-11-07T14:03:16.978ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472024-11-07T14:03:16.978ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482024-11-07T14:03:16.980ZINFOcrucible: Crucible 8200e8ff-5cf5-429e-9bfa-65203d6cde5a has session id: a09782c1-e545-4f8f-9936-41fe5cbb9630 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
492024-11-07T14:03:16.980ZINFOcrucible: Upstairs opts: Upstairs UUID: 8200e8ff-5cf5-429e-9bfa-65203d6cde5a, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
502024-11-07T14:03:16.980ZINFOcrucible: Crucible stats registered with UUID: 8200e8ff-5cf5-429e-9bfa-65203d6cde5a session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
512024-11-07T14:03:17.016ZINFOcrucible: Downstairs in volume = 3 task = crutest
522024-11-07T14:03:17.016ZINFOcrucible: The guest has requested activation with gen:1
532024-11-07T14:03:17.016ZINFOcrucible: 8200e8ff-5cf5-429e-9bfa-65203d6cde5a active request set session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
542024-11-07T14:03:17.016ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
552024-11-07T14:03:17.017ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
562024-11-07T14:03:17.017ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
572024-11-07T14:03:17.017ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
582024-11-07T14:03:17.017ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
592024-11-07T14:03:17.017ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
602024-11-07T14:03:17.017ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
612024-11-07T14:03:17.017ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
622024-11-07T14:03:17.017ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
632024-11-07T14:03:17.017ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46154) = io task client = 0 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
642024-11-07T14:03:17.017ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:65357) = io task client = 2 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
652024-11-07T14:03:17.017ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37864) = io task client = 1 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
662024-11-07T14:03:17.019ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
672024-11-07T14:03:17.019ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
682024-11-07T14:03:17.019ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
692024-11-07T14:03:17.019ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
702024-11-07T14:03:17.019ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
712024-11-07T14:03:17.019ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
722024-11-07T14:03:17.020ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
732024-11-07T14:03:17.020ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
742024-11-07T14:03:17.021ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
752024-11-07T14:03:17.021ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
762024-11-07T14:03:17.021ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
772024-11-07T14:03:17.021ZINFOcrucible: Waiting for more clients to be ready session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
782024-11-07T14:03:17.021ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
792024-11-07T14:03:17.021ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
802024-11-07T14:03:17.021ZINFOcrucible: Waiting for more clients to be ready session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
812024-11-07T14:03:17.021ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
822024-11-07T14:03:17.021ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
832024-11-07T14:03:17.022ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
842024-11-07T14:03:17.022ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
852024-11-07T14:03:17.022ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
862024-11-07T14:03:17.022ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
872024-11-07T14:03:17.022ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
882024-11-07T14:03:17.022ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
892024-11-07T14:03:17.022ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
902024-11-07T14:03:17.022ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
912024-11-07T14:03:17.022ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
922024-11-07T14:03:17.022ZINFOcrucible: Max found gen is 1 = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
932024-11-07T14:03:17.022ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
942024-11-07T14:03:17.022ZINFOcrucible: Next flush: 1 = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
952024-11-07T14:03:17.023ZINFOcrucible: All extents match = downstairs session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
962024-11-07T14:03:17.023ZINFOcrucible: No downstairs reconciliation required session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
972024-11-07T14:03:17.023ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
982024-11-07T14:03:17.023ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
992024-11-07T14:03:17.023ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
1002024-11-07T14:03:17.023ZINFOcrucible: All required reconciliation work is completed session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
1012024-11-07T14:03:17.023ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
1022024-11-07T14:03:17.023ZINFOcrucible: 8200e8ff-5cf5-429e-9bfa-65203d6cde5a is now active with session: a09782c1-e545-4f8f-9936-41fe5cbb9630 session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
1032024-11-07T14:03:17.023ZINFOcrucible: Set Active after no reconciliation session_id = a09782c1-e545-4f8f-9936-41fe5cbb9630
1042024-11-07T14:03:17.023ZINFOcrucible: The guest has finished waiting for activation with:1
105 Activate successful
106 Wait for a query_work_queue command to finish before sending IO
107 Region: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
108 Fill test
109 Read and Verify all blocks (0..5000 range:false)
110 Wrote out file "/tmp/test_replay_verify.log"
111 CLIENT: Tests done. All submitted work has been ACK'd
112 ----------------------------------------------------------------
113 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:1
114 States: Active Active Active
115 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
116 1041 Acked Read 256 Done Done Done false
117 STATES DS:0 DS:1 DS:2 TOTAL
118 Sent 0 0 0 0
119 Done 1 1 1 3
120 Skipped 0 0 0 0
121 Error 0 0 0 0
122 Last Flush: 1040 1040 1040
123 Downstairs last five completed: 1040 1039 1038 1037 1036
124 Upstairs last five completed: 1041 1040 1039 1038 1037
125 CLIENT: Up:0 ds:1 act:3
126 ----------------------------------------------------------------
127 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
128 Downstairs last five completed: 1042 1041 1040 1039 1038
129 Upstairs last five completed: 1042 1041 1040 1039 1038
130 CLIENT: Up:0 ds:0 act:3
131 CLIENT: All crucible jobs finished, exiting program
1322024-11-07T14:03:21.615ZWARNcrucible: Guest handle has been dropped
133 Replay loop starts now November 7, 2024 at 02:03:21 PM UTC
134 Nov 07 14:03:21.689 INFO accepted connection, remote_addr: 127.0.0.1:55494, local_addr: 127.0.0.1:9998
135 Nov 07 14:03:21.690 INFO request completed, latency_us: 180, response_code: 200, uri: /regioninfo, method: GET, req_id: 3395b5d2-b432-4e6f-9e1e-3f2721ed4afc, remote_addr: 127.0.0.1:55494, local_addr: 127.0.0.1:9998
1362024-11-07T14:03:21.690ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
137 Nov 07 14:03:21.691 INFO request completed, latency_us: 118, response_code: 200, uri: /regioncount, method: GET, req_id: 2ef5f17f-921c-407c-967b-c441454ba00d, remote_addr: 127.0.0.1:55494, local_addr: 127.0.0.1:9998
1382024-11-07T14:03:21.691ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
139 Nov 07 14:03:21.691 INFO request completed, latency_us: 121, response_code: 200, uri: /port/cid/0, method: GET, req_id: df475c26-8c83-4aba-8fe0-b41397d6ee8a, remote_addr: 127.0.0.1:55494, local_addr: 127.0.0.1:9998
140 Nov 07 14:03:21.692 INFO request completed, latency_us: 97, response_code: 200, uri: /port/cid/1, method: GET, req_id: 871b950f-445f-4734-a327-50460e5efe41, remote_addr: 127.0.0.1:55494, local_addr: 127.0.0.1:9998
141 Nov 07 14:03:21.692 INFO request completed, latency_us: 91, response_code: 200, uri: /port/cid/2, method: GET, req_id: 9c5e2799-a871-427b-9a50-64ec9f6f6590, remote_addr: 127.0.0.1:55494, local_addr: 127.0.0.1:9998
1422024-11-07T14:03:21.693ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1432024-11-07T14:03:21.693ZINFOcrucible: Upstairs starts
1442024-11-07T14:03:21.693ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "311c3ca2c97d568407ae06780269d96fcfdd3a7c", git_commit_timestamp: "2024-11-07T13:47:04.000000000Z", git_branch: "renovate/tokio-1.x-lockfile", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1452024-11-07T14:03:21.693ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1462024-11-07T14:03:21.693ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1472024-11-07T14:03:21.693ZINFOcrucible: Crucible 51cf14ee-1cee-4219-ae27-6361c0912b37 has session id: ac74b8f4-f0f2-4273-8fc2-10240de40f73 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1482024-11-07T14:03:21.693ZINFOcrucible: Upstairs opts: Upstairs UUID: 51cf14ee-1cee-4219-ae27-6361c0912b37, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1492024-11-07T14:03:21.693ZINFOcrucible: Crucible stats registered with UUID: 51cf14ee-1cee-4219-ae27-6361c0912b37 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1502024-11-07T14:03:21.728ZINFOcrucible: Downstairs in volume = 3 task = crutest
1512024-11-07T14:03:21.728ZINFOcrucible: The guest has requested activation with gen:2
1522024-11-07T14:03:21.728ZINFOcrucible: 51cf14ee-1cee-4219-ae27-6361c0912b37 active request set session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1532024-11-07T14:03:21.728ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1542024-11-07T14:03:21.728ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1552024-11-07T14:03:21.728ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1562024-11-07T14:03:21.728ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1572024-11-07T14:03:21.728ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1582024-11-07T14:03:21.728ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1592024-11-07T14:03:21.728ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1602024-11-07T14:03:21.729ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1612024-11-07T14:03:21.729ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1622024-11-07T14:03:21.729ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36548) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1632024-11-07T14:03:21.729ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34460) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1642024-11-07T14:03:21.729ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52338) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1652024-11-07T14:03:21.729ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1662024-11-07T14:03:21.729ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1672024-11-07T14:03:21.729ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1682024-11-07T14:03:21.729ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1692024-11-07T14:03:21.729ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1702024-11-07T14:03:21.730ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1712024-11-07T14:03:21.730ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1722024-11-07T14:03:21.730ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1732024-11-07T14:03:21.730ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1742024-11-07T14:03:21.731ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1752024-11-07T14:03:21.731ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1762024-11-07T14:03:21.731ZINFOcrucible: Waiting for more clients to be ready session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1772024-11-07T14:03:21.731ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1782024-11-07T14:03:21.731ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1792024-11-07T14:03:21.731ZINFOcrucible: Waiting for more clients to be ready session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1802024-11-07T14:03:21.731ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1812024-11-07T14:03:21.731ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1822024-11-07T14:03:21.731ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1832024-11-07T14:03:21.731ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1842024-11-07T14:03:21.731ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1852024-11-07T14:03:21.731ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1862024-11-07T14:03:21.731ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1872024-11-07T14:03:21.731ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1882024-11-07T14:03:21.731ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1892024-11-07T14:03:21.731ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1902024-11-07T14:03:21.731ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1912024-11-07T14:03:21.731ZINFOcrucible: Max found gen is 2 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1922024-11-07T14:03:21.731ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1932024-11-07T14:03:21.731ZINFOcrucible: Next flush: 2 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1942024-11-07T14:03:21.731ZINFOcrucible: All extents match = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1952024-11-07T14:03:21.731ZINFOcrucible: No downstairs reconciliation required session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
196 {"Activate successful
197 msgWait for a query_work_queue command to finish before sending IO
198 ":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:03:21.731741806Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"client":"0","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
1992024-11-07T14:03:21.731ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2002024-11-07T14:03:21.731ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2012024-11-07T14:03:21.731ZINFOcrucible: All required reconciliation work is completed session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2022024-11-07T14:03:21.731ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2032024-11-07T14:03:21.731ZINFOcrucible: 51cf14ee-1cee-4219-ae27-6361c0912b37 is now active with session: ac74b8f4-f0f2-4273-8fc2-10240de40f73 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2042024-11-07T14:03:21.731ZINFOcrucible: Set Active after no reconciliation session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2052024-11-07T14:03:21.731ZINFOcrucible: The guest has finished waiting for activation with:2
206 Region: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
207 Loading write count information from file "/tmp/test_replay_verify.log"
208 Nov 07 14:03:21.765 INFO accepted connection, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
209 Main task has work to do, go find it
210 got dsc Stop(2)
211 stop 2
212 Nov 07 14:03:21.766 INFO request completed, latency_us: 224, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 4eed069b-8665-4bc3-8b38-2cfa01fb21e5, remote_addr: 127.0.0.1:34100, local_addr: 127.0[8830] Got stop action so:false kr:false
213 .0.1:9998
2142024-11-07T14:03:21.767ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2152024-11-07T14:03:21.767ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2162024-11-07T14:03:21.767ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2172024-11-07T14:03:21.767ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2182024-11-07T14:03:21.767ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
219 Nov 07 14:03:21.767 INFO request completed, latency_us: 246, response_code: 200, uri: /state/cid/2, method: GET, req_id: ce289d9b-3ff9-409d-84e3-82ebd0922c66, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
2202024-11-07T14:03:21.767ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
221 [8830] Exited with: ExitStatus(unix_wait_status(9))
222 [8830][2] reports Exit
223 Nov 07 14:03:25.769 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/2, method: GET, req_id: 5332d0c1-b37a-4d86-8f71-652213e9131a, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
224 001/300 Read block 533 len 16384
225 002/300 Read block 920 len 32768
226 003/300 Write block 3623 len 40960 data: 2 2 2 2 2 2 2 2 2 2
227 004/300 Write block 2017 len 16384 data: 2 2 2 2
228 005/300 Read block 2169 len 24576
229 006/300 Read block 3031 len 28672
230 007/300 Write block 3813 len 32768 data: 2 2 2 2 2 2 2 2
231 008/300 Read block 145 len 32768
232 009/300 Read block 2147 len 36864
233 010/300 Write block 3135 len 8192 data: 2 2
234 011/300 Read block 393 len 28672
235 012/300 Read block 93 len 20480
236 013/300 Read block 192 len 28672
237 014/300 Read block 3988 len 36864
238 015/300 Write block 4693 len 24576 data: 2 2 2 2 2 2
239 016/300 Write block 2201 len 8192 data: 2 2
240 017/300 Read block 1447 len 20480
241 018/300 Write block 4908 len 8192 data: 2 2
242 019/300 Read block 3511 len 4096
243 020/300 Read block 4555 len 36864
244 021/300 Read block 470 len 16384
245 022/300 Flush
246 023/300 Write block 2822 len 28672 data: 2 2 2 2 2 2 2
247 024/300 Read block 1925 len 36864
248 025/300 Write block 3352 len 28672 data: 2 2 2 2 2 2 2
249 026/300 Read block 26 len 12288
250 027/300 Read block 2656 len 36864
251 028/300 Read block 2347 len 16384
252 029/300 Flush
253 030/300 Read block 1686 len 32768
254 031/300 Flush
255 032/300 Read block 2733 len 40960
256 033/300 Read block 217 len 28672
257 034/300 Read block 1295 len 4096
258 035/300 Write block 4842 len 24576 data: 2 2 2 2 2 2
259 036/300 Write block 3147 len 40960 data: 2 2 2 2 2 2 2 2 2 2
260 037/300 Read block 937 len 36864
261 038/300 Read block 2254 len 4096
262 039/300 Read block 3777 len 12288
263 040/300 Read block 1014 len 24576
264 041/300 Write block 2144 len 4096 data: 2
265 042/300 Write block 3618 len 28672 data: 2 2 2 2 2 3 3
266 043/300 Read block 312 len 24576
267 044/300 Read block 1358 len 32768
268 045/300 Flush
269 046/300 Write block 3483 len 8192 data: 2 2
270 047/300 Read block 1479 len 12288
271 048/300 Read block 3584 len 8192
272 049/300 Read block 685 len 32768
273 050/300 Read block 706 len 8192
274 051/300 Write block 2614 len 32768 data: 2 2 2 2 2 2 2 2
275 052/300 Flush
276 053/300 Flush
277 054/300 Write block 3021 len 36864 data: 2 2 2 2 2 2 2 2 2
278 055/300 Flush
279 056/300 Read block 4637 len 4096
280 057/300 Flush
281 058/300 Flush
282 059/300 Flush
283 060/300 Read block 3497 len 36864
284 061/300 Read block 2822 len 16384
285 062/300 Flush
286 063/300 Read block 1828 len 24576
287 064/300 Write block 4754 len 28672 data: 2 2 2 2 2 2 2
288 065/300 Write block 980 len 8192 data: 2 2
289 066/300 Write block 1631 len 4096 data: 2
290 067/300 Read block 2460 len 24576
291 068/300 Write block 543 len 4096 data: 2
292 069/300 Write block 847 len 24576 data: 2 2 2 2 2 2
293 070/300 Write block 980 len 40960 data: 3 3 2 2 2 2 2 2 2 2
294 071/300 Read block 4078 len 8192
295 072/300 Write block 2780 len 24576 data: 2 2 2 2 2 2
296 073/300 Read block 2634 len 8192
297 074/300 Write block 1633 len 40960 data: 2 2 2 2 2 2 2 2 2 2
298 075/300 Read block 780 len 4096
299 076/300 Read block 398 len 40960
300 077/300 Write block 2845 len 16384 data: 2 2 2 2
301 078/300 Read block 671 len 24576
302 079/300 Write block 683 len 8192 data: 2 2
303 080/300 Write block 2833 len 24576 data: 2 2 2 2 2 2
304 081/300 Read block 1714 len 40960
305 082/300 Read block 4128 len 32768
306 083/300 Read block 771 len 24576
307 084/300 Read block 4763 len 12288
308 085/300 Read block 100 len 36864
309 086/300 Read block 687 len 40960
310 087/300 Read block 1993 len 32768
311 088/300 Write block 4628 len 32768 data: 2 2 2 2 2 2 2 2
312 089/300 Read block 539 len 20480
313 090/300 Read block 3071 len 8192
314 091/300 Read block 4606 len 8192
315 092/300 Read block 4759 len 24576
316 093/300 Read block 4398 len 8192
317 094/300 Write block 2002 len 4096 data: 2
318 095/300 Read block 446 len 24576
319 096/300 Read block 2729 len 8192
320 097/300 Write block 1750 len 40960 data: 2 2 2 2 2 2 2 2 2 2
321 098/300 Write block 4189 len 28672 data: 2 2 2 2 2 2 2
322 099/300 Write block 2695 len 24576 data: 2 2 2 2 2 2
323 100/300 Write block 4599 len 8192 data: 2 2
324 101/300 Read block 4748 len 16384
325 102/300 Write block 3167 len 8192 data: 2 2
326 103/300 Read block 2259 len 24576
327 104/300 Read block 87 len 20480
328 105/300 Write block 3821 len 12288 data: 2 2 2
329 106/300 Read block 3618 len 40960
330 107/300 Write block 1720 len 4096 data: 2
331 108/300 Write block 2437 len 40960 data: 2 2 2 2 2 2 2 2 2 2
332 109/300 Read block 1281 len 40960
333 110/300 Write block 1442 len 36864 data: 2 2 2 2 2 2 2 2 2
334 111/300 Read block 2784 len 16384
335 112/300 Write block 3864 len 8192 data: 2 2
336 113/300 Write block 4683 len 32768 data: 2 2 2 2 2 2 2 2
337 114/300 Read block 2800 len 40960
338 115/300 Flush
339 116/300 Read block 3240 len 16384
340 117/300 Write block 2931 len 12288 data: 2 2 2
341 118/300 Read block 4979 len 4096
342 119/300 Write block 914 len 24576 data: 2 2 2 2 2 2
343 120/300 Read block 446 len 4096
344 121/300 Write block 3321 len 16384 data: 2 2 2 2
345 122/300 Write block 3972 len 24576 data: 2 2 2 2 2 2
346 123/300 Write block 4818 len 32768 data: 2 2 2 2 2 2 2 2
347 124/300 Write block 3138 len 4096 data: 2
348 125/300 Read block 3532 len 16384
349 126/300 Write block 3087 len 28672 data: 2 2 2 2 2 2 2
350 127/300 Read block 3843 len 16384
351 128/300 Read block 1180 len 40960
352 129/300 Write block 1806 len 28672 data: 2 2 2 2 2 2 2
353 130/300 Read block 4759 len 32768
354 131/300 Write block 3870 len 24576 data: 2 2 2 2 2 2
355 132/300 Read block 4093 len 12288
356 133/300 Read block 3159 len 24576
357 134/300 Flush
358 135/300 Read block 3968 len 24576
359 136/300 Read block 4068 len 4096
360 137/300 Read block 4931 len 12288
361 138/300 Read block 498 len 8192
362 139/300 Write block 1413 len 36864 data: 2 2 2 2 2 2 2 2 2
363 140/300 Read block 2899 len 4096
364 141/300 Write block 4251 len 24576 data: 2 2 2 2 2 2
365 142/300 Read block 2590 len 40960
366 143/300 Read block 2838 len 28672
367 144/300 Read block 2638 len 24576
368 145/300 Write block 4268 len 24576 data: 2 2 2 2 2 2
369 146/300 Read block 165 len 20480
370 147/300 Write block 3776 len 28672 data: 2 2 2 2 2 2 2
371 148/300 Write block 4597 len 20480 data: 2 2 3 3 2
372 149/300 Read block 4060 len 12288
373 150/300 Write block 1852 len 40960 data: 2 2 2 2 2 2 2 2 2 2
374 151/300 Write block 3604 len 36864 data: 2 2 2 2 2 2 2 2 2
375 152/300 Write block 4534 len 8192 data: 2 2
376 153/300 Write block 436 len 36864 data: 2 2 2 2 2 2 2 2 2
377 154/300 Flush
378 155/300 Read block 349 len 28672
379 156/300 Write block 745 len 32768 data: 2 2 2 2 2 2 2 2
380 157/300 Read block 4575 len 28672
381 158/300 Flush
382 159/300 Write block 3861 len 8192 data: 2 2
383 160/300 Read block 2791 len 32768
384 161/300 Read block 1671 len 28672
385 162/300 Read block 666 len 8192
386 163/300 Write block 3375 len 20480 data: 2 2 2 2 2
387 164/300 Write block 2663 len 16384 data: 2 2 2 2
388 165/300 Read block 583 len 4096
389 166/300 Write block 2075 len 40960 data: 2 2 2 2 2 2 2 2 2 2
390 167/300 Read block 1757 len 16384
391 168/300 Write block 200 len 16384 data: 2 2 2 2
392 169/300 Write block 2682 len 4096 data: 2
393 170/300 Read block 3609 len 12288
394 171/300 Read block 4607 len 28672
395 172/300 Read block 3717 len 28672
396 173/300 Read block 1798 len 12288
397 174/300 Write block 919 len 4096 data: 3
398 175/300 Read block 73 len 12288
399 176/300 Read block 134 len 4096
400 177/300 Write block 4383 len 12288 data: 2 2 2
401 178/300 Write block 2826 len 24576 data: 3 3 3 2 2 2
402 179/300 Read block 4697 len 32768
403 180/300 Flush
404 181/300 Read block 3030 len 20480
405 182/300 Read block 3443 len 20480
406 183/300 Flush
407 184/300 Write block 2340 len 28672 data: 2 2 2 2 2 2 2
408 185/300 Write block 4554 len 28672 data: 2 2 2 2 2 2 2
409 186/300 Read block 4723 len 12288
410 187/300 Write block 828 len 8192 data: 2 2
411 188/300 Read block 450 len 12288
412 189/300 Write block 2005 len 40960 data: 2 2 2 2 2 2 2 2 2 2
413 190/300 Flush
414 191/300 Read block 1602 len 24576
415 192/300 Read block 4985 len 8192
416 193/300 Write block 1673 len 36864 data: 2 2 2 2 2 2 2 2 2
417 194/300 Write block 4447 len 28672 data: 2 2 2 2 2 2 2
418 195/300 Write block 3393 len 20480 data: 2 2 2 2 2
419 196/300 Flush
420 197/300 Write block 466 len 28672 data: 2 2 2 2 2 2 2
421 198/300 Read block 830 len 8192
422 199/300 Read block 4300 len 32768
423 200/300 Write block 1374 len 24576 data: 2 2 2 2 2 2
424 201/300 Read block 2314 len 32768
425 202/300 Read block 1411 len 16384
426 203/300 Read block 37 len 32768
427 204/300 Flush
428 205/300 Read block 964 len 16384
429 206/300 Read block 1665 len 32768
430 207/300 Write block 4237 len 20480 data: 2 2 2 2 2
431 208/300 Write block 3616 len 32768 data: 2 2 3 3 3 3 3 4
432 209/300 Write block 3874 len 40960 data: 3 3 2 2 2 2 2 2 2 2
433 210/300 Write block 342 len 16384 data: 2 2 2 2
434 211/300 Read block 2151 len 28672
435 212/300 Read block 2754 len 36864
436 213/300 Write block 762 len 16384 data: 2 2 2 2
437 214/300 Write block 1678 len 24576 data: 3 3 3 3 2 2
438 215/300 Read block 4328 len 28672
439 216/300 Write block 641 len 32768 data: 2 2 2 2 2 2 2 2
440 217/300 Flush
441 218/300 Write block 482 len 32768 data: 2 2 2 2 2 2 2 2
442 219/300 Read block 1158 len 16384
443 220/300 Write block 727 len 36864 data: 2 2 2 2 2 2 2 2 2
444 221/300 Read block 2746 len 32768
445 222/300 Read block 2905 len 24576
446 223/300 Read block 332 len 8192
447 224/300 Read block 922 len 8192
448 225/300 Write block 2762 len 12288 data: 2 2 2
449 226/300 Flush
450 227/300 Write block 1425 len 40960 data: 2 2 2 2 2 2 2 2 2 2
451 228/300 Write block 1254 len 28672 data: 2 2 2 2 2 2 2
452 229/300 Flush
453 230/300 Write block 468 len 12288 data: 3 3 3
454 231/300 Read block 4464 len 20480
455 232/300 Write block 3321 len 36864 data: 3 3 3 3 2 2 2 2 2
456 233/300 Read block 1831 len 8192
457 234/300 Flush
458 235/300 Read block 309 len 20480
459 236/300 Write block 778 len 20480 data: 2 2 2 2 2
460 237/300 Read block 4926 len 16384
461 238/300 Read block 3296 len 28672
462 239/300 Flush
463 240/300 Flush
464 241/300 Read block 3651 len 8192
465 242/300 Write block 4367 len 36864 data: 2 2 2 2 2 2 2 2 2
466 243/300 Write block 180 len 12288 data: 2 2 2
467 244/300 Read block 4221 len 32768
468 245/300 Write block 4292 len 16384 data: 2 2 2 2
469 246/300 Write block 1850 len 4096 data: 2
470 247/300 Read block 2725 len 16384
471 248/300 Write block 2163 len 40960 data: 2 2 2 2 2 2 2 2 2 2
472 249/300 Write block 1779 len 28672 data: 2 2 2 2 2 2 2
473 250/300 Flush
474 251/300 Read block 826 len 36864
475 252/300 Read block 2400 len 8192
476 253/300 Write block 4541 len 40960 data: 2 2 2 2 2 2 2 2 2 2
477 254/300 Write block 4836 len 36864 data: 2 2 2 2 2 2 3 3 3
478 255/300 Write block 3124 len 20480 data: 2 2 2 2 2
479 256/300 Read block 2327 len 12288
480 257/300 Write block 293 len 12288 data: 2 2 2
481 258/300 Write block 1401 len 40960 data: 2 2 2 2 2 2 2 2 2 2
482 259/300 Read block 2488 len 4096
483 260/300 Flush
484 261/300 Read block 1536 len 24576
485 262/300 Read block 3640 len 40960
486 263/300 Write block 287 len 16384 data: 2 2 2 2
487 264/300 Read block 2407 len 24576
488 265/300 Read block 386 len 4096
489 266/300 Read block 242 len 36864
490 267/300 Write block 3847 len 4096 data: 2
491 268/300 Read block 1350 len 20480
492 269/300 Write block 1644 len 20480 data: 2 2 2 2 2
493 270/300 Write block 450 len 20480 data: 2 2 2 2 2
494 271/300 Read block 2511 len 20480
495 272/300 Write block 46 len 40960 data: 2 2 2 2 2 2 2 2 2 2
496 273/300 Read block 420 len 12288
497 274/300 Write block 151 len 40960 data: 2 2 2 2 2 2 2 2 2 2
498 275/300 Write block 2013 len 40960 data: 3 3 2 2 3 3 3 3 2 2
499 276/300 Flush
500 277/300 Read block 3623 len 8192
501 278/300 Read block 4247 len 12288
502 279/300 Write block 1700 len 32768 data: 2 2 2 2 2 2 2 2
503 280/300 Read block 3720 len 4096
504 281/300 Read block 1528 len 16384
505 282/300 Flush
506 283/300 Write block 1324 len 28672 data: 2 2 2 2 2 2 2
507 284/300 Read block 3792 len 4096
508 285/300 Read block 4455 len 32768
509 286/300 Flush
510 287/300 Read block 2416 len 36864
511 288/300 Write block 4249 len 8192 data: 2 2
512 289/300 Write block 1029 len 12288 data: 2 2 2
513 290/300 Write block 3842 len 12288 data: 2 2 2
514 291/300 Flush
515 292/300 Write block 2916 len 24576 data: 2 2 2 2 2 2
516 293/300 Write block 1121 len 24576 data: 2 2 2 2 2 2
517 294/300 Write block 3492 len 4096 data: 2
518 295/300 Read block 4969 len 36864
519 296/300 Read block 2886 len 40960
520 297/300 Read block 748 len 40960
521 298/300 Read block 876 len 32768
522 299/300 Write block 566 len 20480 data: 2 2 2 2 2
523 300/300 Write block 2446 len 8192 data: 3 2
524 301/300 Read block 4984 len 36864
525 Main task has work to do, go find it
526 got dsc Start(2)
527 start 2
528 Nov 07 14:03:26.071 INFO request completed, latency_us: 236, response_code: 204, uri: /start/cid/2, method: POST, req_id: 6d02367d-3e29-4f73-bcad-88636cb4bbbc, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
529 Replay: started 2, returned:Ok(())
530 ----------------------------------------------------------------
531 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
532 States: Active Active Offline
533 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
534 1000 Acked Read 4 Done Done Sent false
535 1001 Acked Read 8 Done Done Sent false
536 1002 Acked Write 10 Done Done Sent false
537 1003 Acked Write 4 Done Done Sent false
538 1004 Acked Read 6 Done Done Sent false
539 1005 Acked Read 7 Done Done Sent false
540 1006 Acked Write 8 Done Done Sent false
541 1007 Acked Read 8 Done Done Sent false
542 1008 Acked Read 9 Done Done Sent false
543 1009 Acked Write 2 Done Done Sent false
544 1010 Acked Read 7 Done Done Sent false
545 1011 Acked Read 5 Done Done Sent false
546 1012 Acked Read 7 Done Done Sent false
547 1013 Acked Read 9 Done Done Sent false
548 1014 Acked Write 6 Done Done Sent false
549 1015 Acked Write 2 Done Done Sent false
550 1016 Acked Read 5 Done Done Sent false
551 1017 Acked Write 2 Done Done Sent false
552 1018 Acked Read 1 Done Done Sent false
553 1019 Acked Read 9 Done Done Sent false
554 1020 Acked Read 4 Done Done Sent false
555 1021 Acked Flush 0 Done Done Sent false
556 1022 Acked Write 7 Done Done Sent false
557 1023 Acked Read 9 Done Done Sent false
558 1024 Acked Write 7 Done Done Sent false
559 1025 Acked Read 3 Done Done Sent false
560 1026 Acked Read 9 Done Done Sent false
561 1027 Acked Read 4 Done Done Sent false
562 1028 Acked Flush 0 Done Done Sent false
563 1029 Acked Read 8 Done Done Sent false
564 1030 Acked Flush 0 Done Done Sent false
565 1031 Acked Read 10 Done Done Sent false
566 1032 Acked Read 7 Done Done Sent false
567 1033 Acked Read 1 Done Done Sent false
568 1034 Acked Write 6 Done Done Sent false
569 1035 Acked Write 10 Done Done Sent false
570 1036 Acked Read 9 Done Done Sent false
571 1037 Acked Read 1 Done Done Sent false
572 1038 Acked Read 3 Done Done Sent false
573 1039 Acked Read 6 Done Done Sent false
574 1040 Acked Write 1 Done Done Sent false
575 1041 Acked Write 7 Done Done Sent false
576 1042 Acked Read 6 Done Done Sent false
577 1043 Acked Read 8 Done Done Sent false
578 1044 Acked Flush 0 Done Done Sent false
579 1045 Acked Write 2 Done Done Sent false
580 1046 Acked Read 3 Done Done Sent false
581 1047 Acked Read 2 Done Done Sent false
582 1048 Acked Read 8 Done Done Sent false
583 1049 Acked Read 2 Done Done Sent false
584 1050 Acked Write 8 Done Done Sent false
585 1051 Acked Flush 0 Done Done Sent false
586 1052 Acked Flush 0 Done Done Sent false
587 1053 Acked Write 9 Done Done Sent false
588 1054 Acked Flush 0 Done Done Sent false
589 1055 Acked Read 1 Done Done Sent false
590 1056 Acked Flush 0 Done Done Sent false
591 1057 Acked Flush 0 Done Done Sent false
592 1058 Acked Flush 0 Done Done Sent false
593 1059 Acked Read 9 Done Done Sent false
594 1060 Acked Read 4 Done Done Sent false
595 1061 Acked Flush 0 Done Done Sent false
596 1062 Acked Read 6 Done Done Sent false
597 1063 Acked Write 7 Done Done Sent false
598 1064 Acked Write 2 Done Done Sent false
599 1065 Acked Write 1 Done Done Sent false
600 1066 Acked Read 6 Done Done Sent false
601 1067 Acked Write 1 Done Done Sent false
602 1068 Acked Write 6 Done Done Sent false
603 1069 Acked Write 10 Done Done Sent false
604 1070 Acked Read 2 Done Done Sent false
605 1071 Acked Write 6 Done Done Sent false
606 1072 Acked Read 2 Done Done Sent false
607 1073 Acked Write 10 Done Done Sent false
608 1074 Acked Read 1 Done Done Sent false
609 1075 Acked Read 10 Done Done Sent false
610 1076 Acked Write 4 Done Done Sent false
611 1077 Acked Read 6 Done Done Sent false
612 1078 Acked Write 2 Done Done Sent false
613 1079 Acked Write 6 Done Done Sent false
614 1080 Acked Read 10 Done Done Sent false
615 1081 Acked Read 8 Done Done Sent false
616 1082 Acked Read 6 Done Done Sent false
617 1083 Acked Read 3 Done Done Sent false
618 1084 Acked Read 9 Done Done Sent false
619 1085 Acked Read 10 Done Done Sent false
620 1086 Acked Read 8 Done Done Sent false
621 1087 Acked Write 8 Done Done Sent false
622 1088 Acked Read 5 Done Done Sent false
623 1089 Acked Read 2 Done Done Sent false
624 1090 Acked Read 2 Done Done Sent false
625 1091 Acked Read 6 Done Done Sent false
626 1092 Acked Read 2 Done Done Sent false
627 1093 Acked Write 1 Done Done Sent false
628 1094 Acked Read 6 Done Done Sent false
629 1095 Acked Read 2 Done Done Sent false
630 1096 Acked Write 10 Done Done Sent false
631 1097 Acked Write 7 Done Done Sent false
632 1098 Acked Write 6 Done Done Sent false
633 1099 Acked Write 2 Done Done Sent false
634 1100 Acked Read 4 Done Done Sent false
635 1101 Acked Write 2 Done Done Sent false
636 1102 Acked Read 6 Done Done Sent false
637 1103 Acked Read 5 Done Done Sent false
638 1104 Acked Write 3 Done Done Sent false
639 1105 Acked Read 10 Done Done Sent false
640 1106 Acked Write 1 Done Done Sent false
641 1107 Acked Write 10 Done Done Sent false
642 1108 Acked Read 10 Done Done Sent false
643 1109 Acked Write 9 Done Done Sent false
644 1110 Acked Read 4 Done Done Sent false
645 1111 Acked Write 2 Done Done Sent false
646 1112 Acked Write 8 Done Done Sent false
647 1113 Acked Read 10 Done Done Sent false
648 1114 Acked Flush 0 Done Done Sent false
649 1115 Acked Read 4 Done Done Sent false
650 1116 Acked Write 3 Done Done Sent false
651 1117 Acked Read 1 Done Done Sent false
652 1118 Acked Write 6 Done Done Sent false
653 1119 Acked Read 1 Done Done Sent false
654 1120 Acked Write 4 Done Done Sent false
655 1121 Acked Write 6 Done Done Sent false
656 1122 Acked Write 8 Done Done Sent false
657 1123 Acked Write 1 Done Done Sent false
658 1124 Acked Read 4 Done Done Sent false
659 1125 Acked Write 7 Done Done Sent false
660 1126 Acked Read 4 Done Done Sent false
661 1127 Acked Read 10 Done Done Sent false
662 1128 Acked Write 7 Done Done Sent false
663 1129 Acked Read 8 Done Done Sent false
664 1130 Acked Write 6 Done Done Sent false
665 1131 Acked Read 3 Done Done Sent false
666 1132 Acked Read 6 Done Done Sent false
667 1133 Acked Flush 0 Done Done Sent false
668 1134 Acked Read 6 Done Done Sent false
669 1135 Acked Read 1 Done Done Sent false
670 1136 Acked Read 3 Done Done Sent false
671 1137 Acked Read 2 Done Done Sent false
672 1138 Acked Write 9 Done Done Sent false
673 1139 Acked Read 1 Done Done Sent false
674 1140 Acked Write 6 Done Done Sent false
675 1141 Acked Read 10 Done Done Sent false
676 1142 Acked Read 7 Done Done Sent false
677 1143 Acked Read 6 Done Done Sent false
678 1144 Acked Write 6 Done Done Sent false
679 1145 Acked Read 5 Done Done Sent false
680 1146 Acked Write 7 Done Done Sent false
681 1147 Acked Write 5 Done Done Sent false
682 1148 Acked Read 3 Done Done Sent false
683 1149 Acked Write 10 Done Done Sent false
684 1150 Acked Write 9 Done Done Sent false
685 1151 Acked Write 2 Done Done Sent false
686 1152 Acked Write 9 Done Done Sent false
687 1153 Acked Flush 0 Done Done Sent false
688 1154 Acked Read 7 Done Done Sent false
689 1155 Acked Write 8 Done Done Sent false
690 1156 Acked Read 7 Done Done Sent false
691 1157 Acked Flush 0 Done Done Sent false
692 1158 Acked Write 2 Done Done Sent false
693 1159 Acked Read 8 Done Done Sent false
694 1160 Acked Read 7 Done Done Sent false
695 1161 Acked Read 2 Done Done Sent false
696 1162 Acked Write 5 Done Done Sent false
697 1163 Acked Write 4 Done Done Sent false
698 1164 Acked Read 1 Done Done Sent false
699 1165 Acked Write 10 Done Done Sent false
700 1166 Acked Read 4 Done Done Sent false
701 1167 Acked Write 4 Done Done Sent false
702 1168 Acked Write 1 Done Done Sent false
703 1169 Acked Read 3 Done Done Sent false
704 1170 Acked Read 7 Done Done Sent false
705 1171 Acked Read 7 Done Done Sent false
706 1172 Acked Read 3 Done Done Sent false
707 1173 Acked Write 1 Done Done Sent false
708 1174 Acked Read 3 Done Done Sent false
709 1175 Acked Read 1 Done Done Sent false
710 1176 Acked Write 3 Done Done Sent false
711 1177 Acked Write 6 Done Done Sent false
712 1178 Acked Read 8 Done Done Sent false
713 1179 Acked Flush 0 Done Done Sent false
714 1180 Acked Read 5 Done Done Sent false
715 1181 Acked Read 5 Done Done Sent false
716 1182 Acked Flush 0 Done Done Sent false
717 1183 Acked Write 7 Done Done Sent false
718 1184 Acked Write 7 Done Done Sent false
719 1185 Acked Read 3 Done Done Sent false
720 1186 Acked Write 2 Done Done Sent false
721 1187 Acked Read 3 Done Done Sent false
722 1188 Acked Write 10 Done Done Sent false
723 1189 Acked Flush 0 Done Done Sent false
724 1190 Acked Read 6 Done Done Sent false
725 1191 Acked Read 2 Done Done Sent false
726 1192 Acked Write 9 Done Done Sent false
727 1193 Acked Write 7 Done Done Sent false
728 1194 Acked Write 5 Done Done Sent false
729 1195 Acked Flush 0 Done Done Sent false
730 1196 Acked Write 7 Done Done Sent false
731 1197 Acked Read 2 Done Done Sent false
732 1198 Acked Read 8 Done Done Sent false
733 1199 Acked Write 6 Done Done Sent false
734 1200 Acked Read 8 Done Done Sent false
735 1201 Acked Read 4 Done Done Sent false
736 1202 Acked Read 8 Done Done Sent false
737 1203 Acked Flush 0 Done Done Sent false
738 1204 Acked Read 4 Done Done Sent false
739 1205 Acked Read 8 Done Done Sent false
740 1206 Acked Write 5 Done Done Sent false
741 1207 Acked Write 8 Done Done Sent false
742 1208 Acked Write 10 Done Done Sent false
743 1209 Acked Write 4 Done Done Sent false
744 1210 Acked Read 7 Done Done Sent false
745 1211 Acked Read 9 Done Done Sent false
746 1212 Acked Write 4 Done Done Sent false
747 1213 Acked Write 6 Done Done Sent false
748 1214 Acked Read 7 Done Done Sent false
749 1215 Acked Write 8 Done Done Sent false
750 1216 Acked Flush 0 Done Done Sent false
751 1217 Acked Write 8 Done Done Sent false
752 1218 Acked Read 4 Done Done Sent false
753 1219 Acked Write 9 Done Done Sent false
754 1220 Acked Read 8 Done Done Sent false
755 1221 Acked Read 6 Done Done Sent false
756 1222 Acked Read 2 Done Done Sent false
757 1223 Acked Read 2 Done Done Sent false
758 1224 Acked Write 3 Done Done Sent false
759 1225 Acked Flush 0 Done Done Sent false
760 1226 Acked Write 10 Done Done Sent false
761 1227 Acked Write 7 Done Done Sent false
762 1228 Acked Flush 0 Done Done Sent false
763 1229 Acked Write 3 Done Done Sent false
764 1230 Acked Read 5 Done Done Sent false
765 1231 Acked Write 9 Done Done Sent false
766 1232 Acked Read 2 Done Done Sent false
767 1233 Acked Flush 0 Done Done Sent false
768 1234 Acked Read 5 Done Done Sent false
769 1235 Acked Write 5 Done Done Sent false
770 1236 Acked Read 4 Done Done Sent false
771 1237 Acked Read 7 Done Done Sent false
772 1238 Acked Flush 0 Done Done Sent false
773 1239 Acked Flush 0 Done Done Sent false
774 1240 Acked Read 2 Done Done Sent false
775 1241 Acked Write 9 Done Done Sent false
776 1242 Acked Write 3 Done Done Sent false
777 1243 Acked Read 8 Done Done Sent false
778 1244 Acked Write 4 Done Done Sent false
779 1245 Acked Write 1 Done Done Sent false
780 1246 Acked Read 4 Done Done Sent false
781 1247 Acked Write 10 Done Done Sent false
782 1248 Acked Write 7 Done Done Sent false
783 1249 Acked Flush 0 Done Done Sent false
784 1250 Acked Read 9 Done Done Sent false
785 1251 Acked Read 2 Done Done Sent false
786 1252 Acked Write 10 Done Done Sent false
787 1253 Acked Write 9 Done Done Sent false
788 1254 Acked Write 5 Done Done Sent false
789 1255 Acked Read 3 Done Done Sent false
790 1256 Acked Write 3 Done Done Sent false
791 1257 Acked Write 10 Done Done Sent false
792 1258 Acked Read 1 Done Done Sent false
793 1259 Acked Flush 0 Done Done Sent false
794 1260 Acked Read 6 Done Done Sent false
795 1261 Acked Read 10 Done Done Sent false
796 1262 Acked Write 4 Done Done Sent false
797 1263 Acked Read 6 Done Done Sent false
798 1264 Acked Read 1 Done Done Sent false
799 1265 Acked Read 9 Done Done Sent false
800 1266 Acked Write 1 Done Done Sent false
801 1267 Acked Read 5 Done Done Sent false
802 1268 Acked Write 5 Done Done Sent false
803 1269 Acked Write 5 Done Done Sent false
804 1270 Acked Read 5 Done Done Sent false
805 1271 Acked Write 10 Done Done Sent false
806 1272 Acked Read 3 Done Done Sent false
807 1273 Acked Write 10 Done Done Sent false
808 1274 Acked Write 10 Done Done Sent false
809 1275 Acked Flush 0 Done Done Sent false
810 1276 Acked Read 2 Done Done Sent false
811 1277 Acked Read 3 Done Done Sent false
812 1278 Acked Write 8 Done Done Sent false
813 1279 Acked Read 1 Done Done Sent false
814 1280 Acked Read 4 Done Done Sent false
815 1281 Acked Flush 0 Done Done Sent false
816 1282 Acked Write 7 Done Done Sent false
817 1283 Acked Read 1 Done Done Sent false
818 1284 Acked Read 8 Done Done Sent false
819 1285 Acked Flush 0 Done Done Sent false
820 1286 Acked Read 9 Done Done Sent false
821 1287 Acked Write 2 Done Done Sent false
822 1288 Acked Write 3 Done Done Sent false
823 1289 Acked Write 3 Done Done Sent false
824 1290 Acked Flush 0 Done Done Sent false
825 1291 Acked Write 6 Done Done Sent false
826 1292 Acked Write 6 Done Done Sent false
827 1293 Acked Write 1 Done Done Sent false
828 1294 Acked Read 9 Done Done Sent false
829 1295 Acked Read 10 Done Done Sent false
830 1296 Acked Read 10 Done Done Sent false
831 1297 Acked Read 8 Done Done Sent false
832 1298 Acked Write 5 Done Done Sent false
833 1299 Acked Write 2 Done Done Sent false
834 1300 Acked Read 9 Done Done Sent false
835 STATES DS:0 DS:1 DS:2 TOTAL
836 Sent 0 0 301 301
837 Done 301 301 0 602
838 Skipped 0 0 0 0
839 Error 0 0 0 0
840 Last Flush: 1290 1290 0
841 Downstairs last five completed:
842 Upstairs last five completed: 1300 1299 1298 1297 1296
843 CLIENT: Up:0 ds:301 act:2
844 [8830] got start action
845 [8830] I am going to restart
846 Starting downstairs at port 8830
847 Make output file at "/tmp/dsc/downstairs-8830.txt"
848 [8830][2] reports Starting
849 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1200)
850 [8830][2] reports Running
851 ----------------------------------------------------------------
852 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
853 States: Active Active Offline
854 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
855 1000 Acked Read 4 Done Done Sent false
856 1001 Acked Read 8 Done Done Sent false
857 1002 Acked Write 10 Done Done Sent false
858 1003 Acked Write 4 Done Done Sent false
859 1004 Acked Read 6 Done Done Sent false
860 1005 Acked Read 7 Done Done Sent false
861 1006 Acked Write 8 Done Done Sent false
862 1007 Acked Read 8 Done Done Sent false
863 1008 Acked Read 9 Done Done Sent false
864 1009 Acked Write 2 Done Done Sent false
865 1010 Acked Read 7 Done Done Sent false
866 1011 Acked Read 5 Done Done Sent false
867 1012 Acked Read 7 Done Done Sent false
868 1013 Acked Read 9 Done Done Sent false
869 1014 Acked Write 6 Done Done Sent false
870 1015 Acked Write 2 Done Done Sent false
871 1016 Acked Read 5 Done Done Sent false
872 1017 Acked Write 2 Done Done Sent false
873 1018 Acked Read 1 Done Done Sent false
874 1019 Acked Read 9 Done Done Sent false
875 1020 Acked Read 4 Done Done Sent false
876 1021 Acked Flush 0 Done Done Sent false
877 1022 Acked Write 7 Done Done Sent false
878 1023 Acked Read 9 Done Done Sent false
879 1024 Acked Write 7 Done Done Sent false
880 1025 Acked Read 3 Done Done Sent false
881 1026 Acked Read 9 Done Done Sent false
882 1027 Acked Read 4 Done Done Sent false
883 1028 Acked Flush 0 Done Done Sent false
884 1029 Acked Read 8 Done Done Sent false
885 1030 Acked Flush 0 Done Done Sent false
886 1031 Acked Read 10 Done Done Sent false
887 1032 Acked Read 7 Done Done Sent false
888 1033 Acked Read 1 Done Done Sent false
889 1034 Acked Write 6 Done Done Sent false
890 1035 Acked Write 10 Done Done Sent false
891 1036 Acked Read 9 Done Done Sent false
892 1037 Acked Read 1 Done Done Sent false
893 1038 Acked Read 3 Done Done Sent false
894 1039 Acked Read 6 Done Done Sent false
895 1040 Acked Write 1 Done Done Sent false
896 1041 Acked Write 7 Done Done Sent false
897 1042 Acked Read 6 Done Done Sent false
898 1043 Acked Read 8 Done Done Sent false
899 1044 Acked Flush 0 Done Done Sent false
900 1045 Acked Write 2 Done Done Sent false
901 1046 Acked Read 3 Done Done Sent false
902 1047 Acked Read 2 Done Done Sent false
903 1048 Acked Read 8 Done Done Sent false
904 1049 Acked Read 2 Done Done Sent false
905 1050 Acked Write 8 Done Done Sent false
906 1051 Acked Flush 0 Done Done Sent false
907 1052 Acked Flush 0 Done Done Sent false
908 1053 Acked Write 9 Done Done Sent false
909 1054 Acked Flush 0 Done Done Sent false
910 1055 Acked Read 1 Done Done Sent false
911 1056 Acked Flush 0 Done Done Sent false
912 1057 Acked Flush 0 Done Done Sent false
913 1058 Acked Flush 0 Done Done Sent false
914 1059 Acked Read 9 Done Done Sent false
915 1060 Acked Read 4 Done Done Sent false
916 1061 Acked Flush 0 Done Done Sent false
917 1062 Acked Read 6 Done Done Sent false
918 1063 Acked Write 7 Done Done Sent false
919 1064 Acked Write 2 Done Done Sent false
920 1065 Acked Write 1 Done Done Sent false
921 1066 Acked Read 6 Done Done Sent false
922 1067 Acked Write 1 Done Done Sent false
923 1068 Acked Write 6 Done Done Sent false
924 1069 Acked Write 10 Done Done Sent false
925 1070 Acked Read 2 Done Done Sent false
926 1071 Acked Write 6 Done Done Sent false
927 1072 Acked Read 2 Done Done Sent false
928 1073 Acked Write 10 Done Done Sent false
929 1074 Acked Read 1 Done Done Sent false
930 1075 Acked Read 10 Done Done Sent false
931 1076 Acked Write 4 Done Done Sent false
932 1077 Acked Read 6 Done Done Sent false
933 1078 Acked Write 2 Done Done Sent false
934 1079 Acked Write 6 Done Done Sent false
935 1080 Acked Read 10 Done Done Sent false
936 1081 Acked Read 8 Done Done Sent false
937 1082 Acked Read 6 Done Done Sent false
938 1083 Acked Read 3 Done Done Sent false
939 1084 Acked Read 9 Done Done Sent false
940 1085 Acked Read 10 Done Done Sent false
941 1086 Acked Read 8 Done Done Sent false
942 1087 Acked Write 8 Done Done Sent false
943 1088 Acked Read 5 Done Done Sent false
944 1089 Acked Read 2 Done Done Sent false
945 1090 Acked Read 2 Done Done Sent false
946 1091 Acked Read 6 Done Done Sent false
947 1092 Acked Read 2 Done Done Sent false
948 1093 Acked Write 1 Done Done Sent false
949 1094 Acked Read 6 Done Done Sent false
950 1095 Acked Read 2 Done Done Sent false
951 1096 Acked Write 10 Done Done Sent false
952 1097 Acked Write 7 Done Done Sent false
953 1098 Acked Write 6 Done Done Sent false
954 1099 Acked Write 2 Done Done Sent false
955 1100 Acked Read 4 Done Done Sent false
956 1101 Acked Write 2 Done Done Sent false
957 1102 Acked Read 6 Done Done Sent false
958 1103 Acked Read 5 Done Done Sent false
959 1104 Acked Write 3 Done Done Sent false
960 1105 Acked Read 10 Done Done Sent false
961 1106 Acked Write 1 Done Done Sent false
962 1107 Acked Write 10 Done Done Sent false
963 1108 Acked Read 10 Done Done Sent false
964 1109 Acked Write 9 Done Done Sent false
965 1110 Acked Read 4 Done Done Sent false
966 1111 Acked Write 2 Done Done Sent false
967 1112 Acked Write 8 Done Done Sent false
968 1113 Acked Read 10 Done Done Sent false
969 1114 Acked Flush 0 Done Done Sent false
970 1115 Acked Read 4 Done Done Sent false
971 1116 Acked Write 3 Done Done Sent false
972 1117 Acked Read 1 Done Done Sent false
973 1118 Acked Write 6 Done Done Sent false
974 1119 Acked Read 1 Done Done Sent false
975 1120 Acked Write 4 Done Done Sent false
976 1121 Acked Write 6 Done Done Sent false
977 1122 Acked Write 8 Done Done Sent false
978 1123 Acked Write 1 Done Done Sent false
979 1124 Acked Read 4 Done Done Sent false
980 1125 Acked Write 7 Done Done Sent false
981 1126 Acked Read 4 Done Done Sent false
982 1127 Acked Read 10 Done Done Sent false
983 1128 Acked Write 7 Done Done Sent false
984 1129 Acked Read 8 Done Done Sent false
985 1130 Acked Write 6 Done Done Sent false
986 1131 Acked Read 3 Done Done Sent false
987 1132 Acked Read 6 Done Done Sent false
988 1133 Acked Flush 0 Done Done Sent false
989 1134 Acked Read 6 Done Done Sent false
990 1135 Acked Read 1 Done Done Sent false
991 1136 Acked Read 3 Done Done Sent false
992 1137 Acked Read 2 Done Done Sent false
993 1138 Acked Write 9 Done Done Sent false
994 1139 Acked Read 1 Done Done Sent false
995 1140 Acked Write 6 Done Done Sent false
996 1141 Acked Read 10 Done Done Sent false
997 1142 Acked Read 7 Done Done Sent false
998 1143 Acked Read 6 Done Done Sent false
999 1144 Acked Write 6 Done Done Sent false
1000 1145 Acked Read 5 Done Done Sent false
1001 1146 Acked Write 7 Done Done Sent false
1002 1147 Acked Write 5 Done Done Sent false
1003 1148 Acked Read 3 Done Done Sent false
1004 1149 Acked Write 10 Done Done Sent false
1005 1150 Acked Write 9 Done Done Sent false
1006 1151 Acked Write 2 Done Done Sent false
1007 1152 Acked Write 9 Done Done Sent false
1008 1153 Acked Flush 0 Done Done Sent false
1009 1154 Acked Read 7 Done Done Sent false
1010 1155 Acked Write 8 Done Done Sent false
1011 1156 Acked Read 7 Done Done Sent false
1012 1157 Acked Flush 0 Done Done Sent false
1013 1158 Acked Write 2 Done Done Sent false
1014 1159 Acked Read 8 Done Done Sent false
1015 1160 Acked Read 7 Done Done Sent false
1016 1161 Acked Read 2 Done Done Sent false
1017 1162 Acked Write 5 Done Done Sent false
1018 1163 Acked Write 4 Done Done Sent false
1019 1164 Acked Read 1 Done Done Sent false
1020 1165 Acked Write 10 Done Done Sent false
1021 1166 Acked Read 4 Done Done Sent false
1022 1167 Acked Write 4 Done Done Sent false
1023 1168 Acked Write 1 Done Done Sent false
1024 1169 Acked Read 3 Done Done Sent false
1025 1170 Acked Read 7 Done Done Sent false
1026 1171 Acked Read 7 Done Done Sent false
1027 1172 Acked Read 3 Done Done Sent false
1028 1173 Acked Write 1 Done Done Sent false
1029 1174 Acked Read 3 Done Done Sent false
1030 1175 Acked Read 1 Done Done Sent false
1031 1176 Acked Write 3 Done Done Sent false
1032 1177 Acked Write 6 Done Done Sent false
1033 1178 Acked Read 8 Done Done Sent false
1034 1179 Acked Flush 0 Done Done Sent false
1035 1180 Acked Read 5 Done Done Sent false
1036 1181 Acked Read 5 Done Done Sent false
1037 1182 Acked Flush 0 Done Done Sent false
1038 1183 Acked Write 7 Done Done Sent false
1039 1184 Acked Write 7 Done Done Sent false
1040 1185 Acked Read 3 Done Done Sent false
1041 1186 Acked Write 2 Done Done Sent false
1042 1187 Acked Read 3 Done Done Sent false
1043 1188 Acked Write 10 Done Done Sent false
1044 1189 Acked Flush 0 Done Done Sent false
1045 1190 Acked Read 6 Done Done Sent false
1046 1191 Acked Read 2 Done Done Sent false
1047 1192 Acked Write 9 Done Done Sent false
1048 1193 Acked Write 7 Done Done Sent false
1049 1194 Acked Write 5 Done Done Sent false
1050 1195 Acked Flush 0 Done Done Sent false
1051 1196 Acked Write 7 Done Done Sent false
1052 1197 Acked Read 2 Done Done Sent false
1053 1198 Acked Read 8 Done Done Sent false
1054 1199 Acked Write 6 Done Done Sent false
1055 1200 Acked Read 8 Done Done Sent false
1056 1201 Acked Read 4 Done Done Sent false
1057 1202 Acked Read 8 Done Done Sent false
1058 1203 Acked Flush 0 Done Done Sent false
1059 1204 Acked Read 4 Done Done Sent false
1060 1205 Acked Read 8 Done Done Sent false
1061 1206 Acked Write 5 Done Done Sent false
1062 1207 Acked Write 8 Done Done Sent false
1063 1208 Acked Write 10 Done Done Sent false
1064 1209 Acked Write 4 Done Done Sent false
1065 1210 Acked Read 7 Done Done Sent false
1066 1211 Acked Read 9 Done Done Sent false
1067 1212 Acked Write 4 Done Done Sent false
1068 1213 Acked Write 6 Done Done Sent false
1069 1214 Acked Read 7 Done Done Sent false
1070 1215 Acked Write 8 Done Done Sent false
1071 1216 Acked Flush 0 Done Done Sent false
1072 1217 Acked Write 8 Done Done Sent false
1073 1218 Acked Read 4 Done Done Sent false
1074 1219 Acked Write 9 Done Done Sent false
1075 1220 Acked Read 8 Done Done Sent false
1076 1221 Acked Read 6 Done Done Sent false
1077 1222 Acked Read 2 Done Done Sent false
1078 1223 Acked Read 2 Done Done Sent false
1079 1224 Acked Write 3 Done Done Sent false
1080 1225 Acked Flush 0 Done Done Sent false
1081 1226 Acked Write 10 Done Done Sent false
1082 1227 Acked Write 7 Done Done Sent false
1083 1228 Acked Flush 0 Done Done Sent false
1084 1229 Acked Write 3 Done Done Sent false
1085 1230 Acked Read 5 Done Done Sent false
1086 1231 Acked Write 9 Done Done Sent false
1087 1232 Acked Read 2 Done Done Sent false
1088 1233 Acked Flush 0 Done Done Sent false
1089 1234 Acked Read 5 Done Done Sent false
1090 1235 Acked Write 5 Done Done Sent false
1091 1236 Acked Read 4 Done Done Sent false
1092 1237 Acked Read 7 Done Done Sent false
1093 1238 Acked Flush 0 Done Done Sent false
1094 1239 Acked Flush 0 Done Done Sent false
1095 1240 Acked Read 2 Done Done Sent false
1096 1241 Acked Write 9 Done Done Sent false
1097 1242 Acked Write 3 Done Done Sent false
1098 1243 Acked Read 8 Done Done Sent false
1099 1244 Acked Write 4 Done Done Sent false
1100 1245 Acked Write 1 Done Done Sent false
1101 1246 Acked Read 4 Done Done Sent false
1102 1247 Acked Write 10 Done Done Sent false
1103 1248 Acked Write 7 Done Done Sent false
1104 1249 Acked Flush 0 Done Done Sent false
1105 1250 Acked Read 9 Done Done Sent false
1106 1251 Acked Read 2 Done Done Sent false
1107 1252 Acked Write 10 Done Done Sent false
1108 1253 Acked Write 9 Done Done Sent false
1109 1254 Acked Write 5 Done Done Sent false
1110 1255 Acked Read 3 Done Done Sent false
1111 1256 Acked Write 3 Done Done Sent false
1112 1257 Acked Write 10 Done Done Sent false
1113 1258 Acked Read 1 Done Done Sent false
1114 1259 Acked Flush 0 Done Done Sent false
1115 1260 Acked Read 6 Done Done Sent false
1116 1261 Acked Read 10 Done Done Sent false
1117 1262 Acked Write 4 Done Done Sent false
1118 1263 Acked Read 6 Done Done Sent false
1119 1264 Acked Read 1 Done Done Sent false
1120 1265 Acked Read 9 Done Done Sent false
1121 1266 Acked Write 1 Done Done Sent false
1122 1267 Acked Read 5 Done Done Sent false
1123 1268 Acked Write 5 Done Done Sent false
1124 1269 Acked Write 5 Done Done Sent false
1125 1270 Acked Read 5 Done Done Sent false
1126 1271 Acked Write 10 Done Done Sent false
1127 1272 Acked Read 3 Done Done Sent false
1128 1273 Acked Write 10 Done Done Sent false
1129 1274 Acked Write 10 Done Done Sent false
1130 1275 Acked Flush 0 Done Done Sent false
1131 1276 Acked Read 2 Done Done Sent false
1132 1277 Acked Read 3 Done Done Sent false
1133 1278 Acked Write 8 Done Done Sent false
1134 1279 Acked Read 1 Done Done Sent false
1135 1280 Acked Read 4 Done Done Sent false
1136 1281 Acked Flush 0 Done Done Sent false
1137 1282 Acked Write 7 Done Done Sent false
1138 1283 Acked Read 1 Done Done Sent false
1139 1284 Acked Read 8 Done Done Sent false
1140 1285 Acked Flush 0 Done Done Sent false
1141 1286 Acked Read 9 Done Done Sent false
1142 1287 Acked Write 2 Done Done Sent false
1143 1288 Acked Write 3 Done Done Sent false
1144 1289 Acked Write 3 Done Done Sent false
1145 1290 Acked Flush 0 Done Done Sent false
1146 1291 Acked Write 6 Done Done Sent false
1147 1292 Acked Write 6 Done Done Sent false
1148 1293 Acked Write 1 Done Done Sent false
1149 1294 Acked Read 9 Done Done Sent false
1150 1295 Acked Read 10 Done Done Sent false
1151 1296 Acked Read 10 Done Done Sent false
1152 1297 Acked Read 8 Done Done Sent false
1153 1298 Acked Write 5 Done Done Sent false
1154 1299 Acked Write 2 Done Done Sent false
1155 1300 Acked Read 9 Done Done Sent false
1156 1301 Acked Flush 0 Done Done Sent false
1157 STATES DS:0 DS:1 DS:2 TOTAL
1158 Sent 0 0 302 302
1159 Done 302 302 0 604
1160 Skipped 0 0 0 0
1161 Error 0 0 0 0
1162 Last Flush: 1301 1301 0
1163 Downstairs last five completed:
1164 Upstairs last five completed: 1301 1300 1299 1298 1297
1165 CLIENT: Up:0 ds:302 act:2
11662024-11-07T14:03:31.769ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11672024-11-07T14:03:31.769ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40740) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11682024-11-07T14:03:31.770ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11692024-11-07T14:03:31.770ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11702024-11-07T14:03:31.770ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11712024-11-07T14:03:31.770ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11722024-11-07T14:03:31.771ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11732024-11-07T14:03:31.771ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11742024-11-07T14:03:31.771ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11752024-11-07T14:03:31.771ZINFOcrucible: [2] client re-new 302 jobs since flush 0 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11762024-11-07T14:03:31.771ZINFOcrucible: [2] Replayed 302 jobs since flush: 0 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1177 ----------------------------------------------------------------
1178 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1179 Downstairs last five completed: 1301 1300 1299 1298 1297
1180 Upstairs last five completed: 1301 1300 1299 1298 1297
1181 CLIENT: Up:0 ds:0 act:3
1182 Replay: All jobs finished
1183 Main task has work to do, go find it
1184 got dsc Stop(0)
1185 stop 0
1186 Nov 07 14:03:34.076 INFO request completed, latency_us: 363, response_code: 204, uri: /stop/cid/0, [8810] Got stop action so:false kr:false
1187 method: POST, req_id: 2f22fe10-2a17-4f97-8af2-6169831f1d4d, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
11882024-11-07T14:03:34.077ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11892024-11-07T14:03:34.077ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11902024-11-07T14:03:34.077ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11912024-11-07T14:03:34.077ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11922024-11-07T14:03:34.077ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11932024-11-07T14:03:34.077ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
1194 Nov 07 14:03:34.077 INFO request completed, latency_us: 275, response_code: 200, uri: /state/cid/0, method: GET, req_id: 08459412-3a28-477c-b5e6-1304dee3d3b8, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
1195 [8810] Exited with: ExitStatus(unix_wait_status(9))
1196 [8810][0] reports Exit
1197 Nov 07 14:03:38.079 INFO request completed, latency_us: 219, response_code: 200, uri: /state/cid/0, method: GET, req_id: 991f7040-cfa9-4a24-937a-99495849db2a, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
1198 001/300 Write block 3923 len 24576 data: 2 2 2 2 2 2
1199 002/300 Flush
1200 003/300 Read block 4450 len 36864
1201 004/300 Read block 998 len 20480
1202 005/300 Read block 2230 len 12288
1203 006/300 Read block 3739 len 12288
1204 007/300 Write block 2998 len 8192 data: 2 2
1205 008/300 Read block 2625 len 16384
1206 009/300 Write block 2817 len 20480 data: 2 2 2 2 2
1207 010/300 Write block 1572 len 20480 data: 2 2 2 2 2
1208 011/300 Write block 2539 len 24576 data: 2 2 2 2 2 2
1209 012/300 Read block 1720 len 24576
1210 013/300 Write block 324 len 8192 data: 2 2
1211 014/300 Write block 452 len 24576 data: 3 3 3 2 2 2
1212 015/300 Read block 3128 len 4096
1213 016/300 Flush
1214 017/300 Flush
1215 018/300 Read block 4536 len 24576
1216 019/300 Write block 4906 len 28672 data: 2 2 3 3 2 2 2
1217 020/300 Read block 634 len 32768
1218 021/300 Write block 3558 len 8192 data: 2 2
1219 022/300 Flush
1220 023/300 Read block 3205 len 12288
1221 024/300 Read block 1562 len 20480
1222 025/300 Write block 1089 len 24576 data: 2 2 2 2 2 2
1223 026/300 Read block 372 len 20480
1224 027/300 Read block 4059 len 4096
1225 028/300 Read block 410 len 12288
1226 029/300 Write block 3248 len 12288 data: 2 2 2
1227 030/300 Flush
1228 031/300 Read block 4349 len 4096
1229 032/300 Flush
1230 033/300 Read block 3498 len 24576
1231 034/300 Read block 3102 len 16384
1232 035/300 Write block 3335 len 8192 data: 2 2
1233 036/300 Read block 1091 len 12288
1234 037/300 Read block 2235 len 24576
1235 038/300 Read block 1668 len 32768
1236 039/300 Read block 1376 len 24576
1237 040/300 Read block 898 len 4096
1238 041/300 Read block 4177 len 16384
1239 042/300 Write block 2978 len 4096 data: 2
1240 043/300 Read block 4007 len 8192
1241 044/300 Write block 1000 len 16384 data: 2 2 2 2
1242 045/300 Read block 871 len 32768
1243 046/300 Read block 3770 len 16384
1244 047/300 Write block 395 len 20480 data: 2 2 2 2 2
1245 048/300 Read block 3928 len 4096
1246 049/300 Read block 754 len 4096
1247 050/300 Read block 972 len 16384
1248 051/300 Write block 4770 len 20480 data: 2 2 2 2 2
1249 052/300 Write block 1794 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1250 053/300 Read block 2099 len 12288
1251 054/300 Read block 4969 len 16384
1252 055/300 Flush
1253 056/300 Read block 2698 len 4096
1254 057/300 Read block 4200 len 28672
1255 058/300 Read block 487 len 40960
1256 059/300 Read block 949 len 8192
1257 060/300 Read block 2878 len 8192
1258 061/300 Read block 2932 len 16384
1259 062/300 Write block 2504 len 4096 data: 2
1260 063/300 Write block 254 len 12288 data: 2 2 2
1261 064/300 Read block 1804 len 40960
1262 065/300 Read block 2982 len 16384
1263 066/300 Write block 160 len 40960 data: 3 2 2 2 2 2 2 2 2 2
1264 067/300 Flush
1265 068/300 Read block 3387 len 28672
1266 069/300 Read block 1401 len 12288
1267 070/300 Read block 4579 len 28672
1268 071/300 Read block 54 len 8192
1269 072/300 Write block 2494 len 20480 data: 2 2 2 2 2
1270 073/300 Write block 4994 len 4096 data: 2
1271 074/300 Read block 2519 len 36864
1272 075/300 Read block 1513 len 12288
1273 076/300 Write block 3208 len 36864 data: 2 2 2 2 2 2 2 2 2
1274 077/300 Write block 4366 len 28672 data: 2 3 3 3 3 3 3
1275 078/300 Write block 3230 len 36864 data: 2 2 2 2 2 2 2 2 2
1276 079/300 Read block 4532 len 36864
1277 080/300 Read block 3317 len 32768
1278 081/300 Read block 1301 len 24576
1279 082/300 Read block 2292 len 24576
1280 083/300 Read block 1668 len 12288
1281 084/300 Write block 4586 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1282 085/300 Read block 1156 len 12288
1283 086/300 Write block 3895 len 36864 data: 2 2 2 2 2 2 2 2 2
1284 087/300 Write block 3394 len 8192 data: 3 3
1285 088/300 Read block 4902 len 36864
1286 089/300 Read block 1528 len 40960
1287 090/300 Write block 3618 len 20480 data: 4 4 4 4 4
1288 091/300 Flush
1289 092/300 Read block 4587 len 40960
1290 093/300 Write block 2854 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1291 094/300 Write block 4355 len 32768 data: 2 2 2 2 2 2 2 2
1292 095/300 Read block 4031 len 4096
1293 096/300 Read block 1166 len 36864
1294 097/300 Write block 3140 len 12288 data: 2 2 2
1295 098/300 Flush
1296 099/300 Write block 3339 len 28672 data: 2 2 2 2 2 2 2
1297 100/300 Read block 505 len 24576
1298 101/300 Flush
1299 102/300 Flush
1300 103/300 Write block 610 len 20480 data: 2 2 2 2 2
1301 104/300 Write block 2064 len 8192 data: 2 2
1302 105/300 Read block 4940 len 40960
1303 106/300 Write block 3218 len 12288 data: 2 2 2
1304 107/300 Read block 3540 len 36864
1305 108/300 Read block 3865 len 4096
1306 109/300 Write block 2032 len 8192 data: 2 2
1307 110/300 Read block 1278 len 12288
1308 111/300 Read block 4864 len 8192
1309 112/300 Read block 1559 len 24576
1310 113/300 Write block 4753 len 24576 data: 2 3 3 3 3 3
1311 114/300 Write block 3930 len 24576 data: 2 2 2 2 2 2
1312 115/300 Read block 2723 len 28672
1313 116/300 Flush
1314 117/300 Write block 4010 len 28672 data: 2 2 2 2 2 2 2
1315 118/300 Read block 1409 len 4096
1316 119/300 Write block 863 len 32768 data: 2 2 2 2 2 2 2 2
1317 120/300 Read block 4161 len 12288
1318 121/300 Read block 4962 len 24576
1319 122/300 Read block 2343 len 16384
1320 123/300 Write block 1702 len 16384 data: 3 3 3 3
1321 124/300 Read block 2916 len 20480
1322 125/300 Flush
1323 126/300 Read block 3988 len 28672
1324 127/300 Read block 2112 len 40960
1325 128/300 Write block 593 len 24576 data: 2 2 2 2 2 2
1326 129/300 Write block 949 len 36864 data: 2 2 2 2 2 2 2 2 2
1327 130/300 Read block 4310 len 20480
1328 131/300 Write block 887 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1329 132/300 Write block 4741 len 24576 data: 2 2 2 2 2 2
1330 133/300 Write block 1439 len 24576 data: 2 2 2 3 3 3
1331 134/300 Write block 4556 len 20480 data: 3 3 3 3 3
1332 135/300 Read block 4856 len 28672
1333 136/300 Write block 4658 len 8192 data: 2 2
1334 137/300 Write block 4583 len 4096 data: 2
1335 138/300 Read block 2105 len 12288
1336 139/300 Flush
1337 140/300 Read block 1177 len 36864
1338 141/300 Read block 4260 len 24576
1339 142/300 Write block 12 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1340 143/300 Read block 3837 len 36864
1341 144/300 Write block 3424 len 32768 data: 2 2 2 2 2 2 2 2
1342 145/300 Read block 2619 len 24576
1343 146/300 Read block 1532 len 36864
1344 147/300 Read block 1517 len 32768
1345 148/300 Read block 2991 len 8192
1346 149/300 Read block 1479 len 16384
1347 150/300 Write block 3446 len 4096 data: 2
1348 151/300 Flush
1349 152/300 Write block 2737 len 12288 data: 2 2 2
1350 153/300 Read block 2150 len 36864
1351 154/300 Read block 1479 len 28672
1352 155/300 Read block 637 len 4096
1353 156/300 Write block 3575 len 12288 data: 2 2 2
1354 157/300 Read block 1921 len 16384
1355 158/300 Read block 3932 len 4096
1356 159/300 Flush
1357 160/300 Read block 2144 len 24576
1358 161/300 Read block 3418 len 40960
1359 162/300 Write block 312 len 12288 data: 2 2 2
1360 163/300 Write block 2304 len 24576 data: 2 2 2 2 2 2
1361 164/300 Write block 704 len 28672 data: 2 2 2 2 2 2 2
1362 165/300 Read block 2308 len 12288
1363 166/300 Write block 1078 len 28672 data: 2 2 2 2 2 2 2
1364 167/300 Read block 582 len 36864
1365 168/300 Read block 3463 len 8192
1366 169/300 Read block 2270 len 8192
1367 170/300 Flush
1368 171/300 Read block 645 len 20480
1369 172/300 Read block 579 len 4096
1370 173/300 Read block 856 len 40960
1371 174/300 Read block 3557 len 28672
1372 175/300 Write block 547 len 32768 data: 2 2 2 2 2 2 2 2
1373 176/300 Read block 2470 len 4096
1374 177/300 Write block 2824 len 16384 data: 3 3 4 4
1375 178/300 Read block 1108 len 4096
1376 179/300 Flush
1377 180/300 Flush
1378 181/300 Write block 2157 len 12288 data: 2 2 2
1379 182/300 Write block 1080 len 16384 data: 3 3 3 3
1380 183/300 Read block 1357 len 24576
1381 184/300 Write block 2911 len 8192 data: 2 2
1382 185/300 Read block 3354 len 20480
1383 186/300 Flush
1384 187/300 Write block 1841 len 12288 data: 2 2 2
1385 188/300 Write block 2658 len 32768 data: 2 2 2 2 2 3 3 3
1386 189/300 Read block 4452 len 40960
1387 190/300 Read block 4551 len 32768
1388 191/300 Read block 3598 len 4096
1389 192/300 Read block 725 len 40960
1390 193/300 Read block 3600 len 16384
1391 194/300 Read block 250 len 12288
1392 195/300 Read block 1699 len 12288
1393 196/300 Write block 3657 len 12288 data: 2 2 2
1394 197/300 Read block 587 len 16384
1395 198/300 Flush
1396 199/300 Read block 796 len 28672
1397 200/300 Read block 2553 len 28672
1398 201/300 Read block 2451 len 32768
1399 202/300 Write block 4123 len 8192 data: 2 2
1400 203/300 Flush
1401 204/300 Read block 2365 len 4096
1402 205/300 Write block 1341 len 20480 data: 2 2 2 2 2
1403 206/300 Flush
1404 207/300 Read block 892 len 8192
1405 208/300 Write block 598 len 40960 data: 3 2 2 2 2 2 2 2 2 2
1406 209/300 Write block 4161 len 8192 data: 2 2
1407 210/300 Write block 642 len 24576 data: 3 3 3 3 3 3
1408 211/300 Write block 3056 len 4096 data: 2
1409 212/300 Read block 4816 len 16384
1410 213/300 Read block 3896 len 24576
1411 214/300 Read block 3286 len 4096
1412 215/300 Read block 4162 len 28672
1413 216/300 Write block 184 len 28672 data: 2 2 2 2 2 2 2
1414 217/300 Read block 3185 len 20480
1415 218/300 Flush
1416 219/300 Write block 721 len 32768 data: 2 2 2 2 2 2 3 3
1417 220/300 Read block 1191 len 20480
1418 221/300 Write block 4973 len 32768 data: 2 2 2 2 2 2 2 2
1419 222/300 Read block 1473 len 8192
1420 223/300 Flush
1421 224/300 Read block 3039 len 32768
1422 225/300 Read block 4094 len 20480
1423 226/300 Read block 2299 len 12288
1424 227/300 Write block 3669 len 8192 data: 2 2
1425 228/300 Read block 4390 len 20480
1426 229/300 Read block 1911 len 12288
1427 230/300 Write block 1992 len 4096 data: 2
1428 231/300 Read block 4979 len 36864
1429 232/300 Read block 4279 len 8192
1430 233/300 Flush
1431 234/300 Read block 139 len 16384
1432 235/300 Read block 1778 len 20480
1433 236/300 Read block 4916 len 20480
1434 237/300 Read block 3601 len 16384
1435 238/300 Write block 4755 len 4096 data: 4
1436 239/300 Read block 4348 len 36864
1437 240/300 Write block 567 len 40960 data: 3 3 3 3 2 2 2 2 2 2
1438 241/300 Read block 3398 len 20480
1439 242/300 Write block 4119 len 40960 data: 2 2 2 2 3 3 2 2 2 2
1440 243/300 Write block 3828 len 36864 data: 2 2 2 2 2 2 2 2 2
1441 244/300 Write block 1075 len 36864 data: 2 2 2 3 3 4 4 4 4
1442 245/300 Read block 4660 len 32768
1443 246/300 Write block 580 len 32768 data: 2 2 2 2 2 2 2 2
1444 247/300 Read block 2934 len 16384
1445 248/300 Write block 3399 len 24576 data: 2 2 2 2 2 2
1446 249/300 Read block 513 len 12288
1447 250/300 Read block 280 len 36864
1448 251/300 Write block 4153 len 4096 data: 2
1449 252/300 Read block 2096 len 40960
1450 253/300 Read block 557 len 12288
1451 254/300 Flush
1452 255/300 Read block 3041 len 40960
1453 256/300 Read block 530 len 16384
1454 257/300 Read block 1954 len 12288
1455 258/300 Write block 2083 len 4096 data: 3
1456 259/300 Read block 2860 len 20480
1457 260/300 Read block 2012 len 24576
1458 261/300 Write block 2423 len 24576 data: 2 2 2 2 2 2
1459 262/300 Read block 2504 len 8192
1460 263/300 Write block 1786 len 40960 data: 2 2 2 2 2 2 2 2 3 3
1461 264/300 Flush
1462 265/300 Write block 732 len 8192 data: 3 3
1463 266/300 Write block 3081 len 32768 data: 2 2 2 2 2 2 3 3
1464 267/300 Write block 2801 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1465 268/300 Write block 3924 len 20480 data: 3 3 3 3 3
1466 269/300 Write block 428 len 20480 data: 2 2 2 2 2
1467 270/300 Read block 2075 len 24576
1468 271/300 Read block 1229 len 32768
1469 272/300 Write block 2494 len 12288 data: 3 3 3
1470 273/300 Write block 341 len 12288 data: 2 3 3
1471 274/300 Read block 354 len 24576
1472 275/300 Read block 1969 len 36864
1473 276/300 Write block 704 len 20480 data: 3 3 3 3 3
1474 277/300 Read block 4621 len 28672
1475 278/300 Read block 2775 len 12288
1476 279/300 Write block 1521 len 8192 data: 2 2
1477 280/300 Read block 1095 len 24576
1478 281/300 Read block 1012 len 36864
1479 282/300 Write block 1611 len 4096 data: 2
1480 283/300 Read block 1322 len 28672
1481 284/300 Flush
1482 285/300 Write block 428 len 24576 data: 3 3 3 3 3 2
1483 286/300 Read block 2556 len 20480
1484 287/300 Read block 2569 len 4096
1485 288/300 Write block 1442 len 36864 data: 4 4 4 3 3 3 3 3 3
1486 289/300 Read block 4758 len 16384
1487 290/300 Read block 2804 len 12288
1488 291/300 Write block 4146 len 36864 data: 2 2 2 2 2 2 2 3 2
1489 292/300 Write block 3105 len 16384 data: 2 2 2 2
1490 293/300 Flush
1491 294/300 Read block 4988 len 8192
1492 295/300 Write block 4432 len 32768 data: 2 2 2 2 2 2 2 2
1493 296/300 Write block 20 len 36864 data: 3 3 2 2 2 2 2 2 2
1494 297/300 Write block 3282 len 8192 data: 2 2
1495 298/300 Write block 695 len 12288 data: 2 2 2
1496 299/300 Write block 3272 len 16384 data: 2 2 2 2
1497 300/300 Read block 1091 len 20480
1498 301/300 Write block 1461 len 32768 data: 2 2 2 2 2 2 2 2
1499 Main task has work to do, go find it
1500 got dsc Start(0)
1501 start 0
1502 Nov 07 14:03:38.368 INFO request completed, latency_us: 244, response_code: 204, uri: /start/cid/0, method: POST, req_id: de9ee573-897d-4cd3-9c41-063df1739e2d, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
1503 Replay: started 0, returned:Ok(())
1504 ----------------------------------------------------------------
1505 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1506 States: Offline Active Active
1507 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1508 1302 Acked Write 6 Sent Done Done false
1509 1303 Acked Flush 0 Sent Done Done false
1510 1304 Acked Read 9 Sent Done Done false
1511 1305 Acked Read 5 Sent Done Done false
1512 1306 Acked Read 3 Sent Done Done false
1513 1307 Acked Read 3 Sent Done Done false
1514 1308 Acked Write 2 Sent Done Done false
1515 1309 Acked Read 4 Sent Done Done false
1516 1310 Acked Write 5 Sent Done Done false
1517 1311 Acked Write 5 Sent Done Done false
1518 1312 Acked Write 6 Sent Done Done false
1519 1313 Acked Read 6 Sent Done Done false
1520 1314 Acked Write 2 Sent Done Done false
1521 1315 Acked Write 6 Sent Done Done false
1522 1316 Acked Read 1 Sent Done Done false
1523 1317 Acked Flush 0 Sent Done Done false
1524 1318 Acked Flush 0 Sent Done Done false
1525 1319 Acked Read 6 Sent Done Done false
1526 1320 Acked Write 7 Sent Done Done false
1527 1321 Acked Read 8 Sent Done Done false
1528 1322 Acked Write 2 Sent Done Done false
1529 1323 Acked Flush 0 Sent Done Done false
1530 1324 Acked Read 3 Sent Done Done false
1531 1325 Acked Read 5 Sent Done Done false
1532 1326 Acked Write 6 Sent Done Done false
1533 1327 Acked Read 5 Sent Done Done false
1534 1328 Acked Read 1 Sent Done Done false
1535 1329 Acked Read 3 Sent Done Done false
1536 1330 Acked Write 3 Sent Done Done false
1537 1331 Acked Flush 0 Sent Done Done false
1538 1332 Acked Read 1 Sent Done Done false
1539 1333 Acked Flush 0 Sent Done Done false
1540 1334 Acked Read 6 Sent Done Done false
1541 1335 Acked Read 4 Sent Done Done false
1542 1336 Acked Write 2 Sent Done Done false
1543 1337 Acked Read 3 Sent Done Done false
1544 1338 Acked Read 6 Sent Done Done false
1545 1339 Acked Read 8 Sent Done Done false
1546 1340 Acked Read 6 Sent Done Done false
1547 1341 Acked Read 1 Sent Done Done false
1548 1342 Acked Read 4 Sent Done Done false
1549 1343 Acked Write 1 Sent Done Done false
1550 1344 Acked Read 2 Sent Done Done false
1551 1345 Acked Write 4 Sent Done Done false
1552 1346 Acked Read 8 Sent Done Done false
1553 1347 Acked Read 4 Sent Done Done false
1554 1348 Acked Write 5 Sent Done Done false
1555 1349 Acked Read 1 Sent Done Done false
1556 1350 Acked Read 1 Sent Done Done false
1557 1351 Acked Read 4 Sent Done Done false
1558 1352 Acked Write 5 Sent Done Done false
1559 1353 Acked Write 10 Sent Done Done false
1560 1354 Acked Read 3 Sent Done Done false
1561 1355 Acked Read 4 Sent Done Done false
1562 1356 Acked Flush 0 Sent Done Done false
1563 1357 Acked Read 1 Sent Done Done false
1564 1358 Acked Read 7 Sent Done Done false
1565 1359 Acked Read 10 Sent Done Done false
1566 1360 Acked Read 2 Sent Done Done false
1567 1361 Acked Read 2 Sent Done Done false
1568 1362 Acked Read 4 Sent Done Done false
1569 1363 Acked Write 1 Sent Done Done false
1570 1364 Acked Write 3 Sent Done Done false
1571 1365 Acked Read 10 Sent Done Done false
1572 1366 Acked Read 4 Sent Done Done false
1573 1367 Acked Write 10 Sent Done Done false
1574 1368 Acked Flush 0 Sent Done Done false
1575 1369 Acked Read 7 Sent Done Done false
1576 1370 Acked Read 3 Sent Done Done false
1577 1371 Acked Read 7 Sent Done Done false
1578 1372 Acked Read 2 Sent Done Done false
1579 1373 Acked Write 5 Sent Done Done false
1580 1374 Acked Write 1 Sent Done Done false
1581 1375 Acked Read 9 Sent Done Done false
1582 1376 Acked Read 3 Sent Done Done false
1583 1377 Acked Write 9 Sent Done Done false
1584 1378 Acked Write 7 Sent Done Done false
1585 1379 Acked Write 9 Sent Done Done false
1586 1380 Acked Read 9 Sent Done Done false
1587 1381 Acked Read 8 Sent Done Done false
1588 1382 Acked Read 6 Sent Done Done false
1589 1383 Acked Read 6 Sent Done Done false
1590 1384 Acked Read 3 Sent Done Done false
1591 1385 Acked Write 10 Sent Done Done false
1592 1386 Acked Read 3 Sent Done Done false
1593 1387 Acked Write 9 Sent Done Done false
1594 1388 Acked Write 2 Sent Done Done false
1595 1389 Acked Read 9 Sent Done Done false
1596 1390 Acked Read 10 Sent Done Done false
1597 1391 Acked Write 5 Sent Done Done false
1598 1392 Acked Flush 0 Sent Done Done false
1599 1393 Acked Read 10 Sent Done Done false
1600 1394 Acked Write 10 Sent Done Done false
1601 1395 Acked Write 8 Sent Done Done false
1602 1396 Acked Read 1 Sent Done Done false
1603 1397 Acked Read 9 Sent Done Done false
1604 1398 Acked Write 3 Sent Done Done false
1605 1399 Acked Flush 0 Sent Done Done false
1606 1400 Acked Write 7 Sent Done Done false
1607 1401 Acked Read 6 Sent Done Done false
1608 1402 Acked Flush 0 Sent Done Done false
1609 1403 Acked Flush 0 Sent Done Done false
1610 1404 Acked Write 5 Sent Done Done false
1611 1405 Acked Write 2 Sent Done Done false
1612 1406 Acked Read 10 Sent Done Done false
1613 1407 Acked Write 3 Sent Done Done false
1614 1408 Acked Read 9 Sent Done Done false
1615 1409 Acked Read 1 Sent Done Done false
1616 1410 Acked Write 2 Sent Done Done false
1617 1411 Acked Read 3 Sent Done Done false
1618 1412 Acked Read 2 Sent Done Done false
1619 1413 Acked Read 6 Sent Done Done false
1620 1414 Acked Write 6 Sent Done Done false
1621 1415 Acked Write 6 Sent Done Done false
1622 1416 Acked Read 7 Sent Done Done false
1623 1417 Acked Flush 0 Sent Done Done false
1624 1418 Acked Write 7 Sent Done Done false
1625 1419 Acked Read 1 Sent Done Done false
1626 1420 Acked Write 8 Sent Done Done false
1627 1421 Acked Read 3 Sent Done Done false
1628 1422 Acked Read 6 Sent Done Done false
1629 1423 Acked Read 4 Sent Done Done false
1630 1424 Acked Write 4 Sent Done Done false
1631 1425 Acked Read 5 Sent Done Done false
1632 1426 Acked Flush 0 Sent Done Done false
1633 1427 Acked Read 7 Sent Done Done false
1634 1428 Acked Read 10 Sent Done Done false
1635 1429 Acked Write 6 Sent Done Done false
1636 1430 Acked Write 9 Sent Done Done false
1637 1431 Acked Read 5 Sent Done Done false
1638 1432 Acked Write 10 Sent Done Done false
1639 1433 Acked Write 6 Sent Done Done false
1640 1434 Acked Write 6 Sent Done Done false
1641 1435 Acked Write 5 Sent Done Done false
1642 1436 Acked Read 7 Sent Done Done false
1643 1437 Acked Write 2 Sent Done Done false
1644 1438 Acked Write 1 Sent Done Done false
1645 1439 Acked Read 3 Sent Done Done false
1646 1440 Acked Flush 0 Sent Done Done false
1647 1441 Acked Read 9 Sent Done Done false
1648 1442 Acked Read 6 Sent Done Done false
1649 1443 Acked Write 10 Sent Done Done false
1650 1444 Acked Read 9 Sent Done Done false
1651 1445 Acked Write 8 Sent Done Done false
1652 1446 Acked Read 6 Sent Done Done false
1653 1447 Acked Read 9 Sent Done Done false
1654 1448 Acked Read 8 Sent Done Done false
1655 1449 Acked Read 2 Sent Done Done false
1656 1450 Acked Read 4 Sent Done Done false
1657 1451 Acked Write 1 Sent Done Done false
1658 1452 Acked Flush 0 Sent Done Done false
1659 1453 Acked Write 3 Sent Done Done false
1660 1454 Acked Read 9 Sent Done Done false
1661 1455 Acked Read 7 Sent Done Done false
1662 1456 Acked Read 1 Sent Done Done false
1663 1457 Acked Write 3 Sent Done Done false
1664 1458 Acked Read 4 Sent Done Done false
1665 1459 Acked Read 1 Sent Done Done false
1666 1460 Acked Flush 0 Sent Done Done false
1667 1461 Acked Read 6 Sent Done Done false
1668 1462 Acked Read 10 Sent Done Done false
1669 1463 Acked Write 3 Sent Done Done false
1670 1464 Acked Write 6 Sent Done Done false
1671 1465 Acked Write 7 Sent Done Done false
1672 1466 Acked Read 3 Sent Done Done false
1673 1467 Acked Write 7 Sent Done Done false
1674 1468 Acked Read 9 Sent Done Done false
1675 1469 Acked Read 2 Sent Done Done false
1676 1470 Acked Read 2 Sent Done Done false
1677 1471 Acked Flush 0 Sent Done Done false
1678 1472 Acked Read 5 Sent Done Done false
1679 1473 Acked Read 1 Sent Done Done false
1680 1474 Acked Read 10 Sent Done Done false
1681 1475 Acked Read 7 Sent Done Done false
1682 1476 Acked Write 8 Sent Done Done false
1683 1477 Acked Read 1 Sent Done Done false
1684 1478 Acked Write 4 Sent Done Done false
1685 1479 Acked Read 1 Sent Done Done false
1686 1480 Acked Flush 0 Sent Done Done false
1687 1481 Acked Flush 0 Sent Done Done false
1688 1482 Acked Write 3 Sent Done Done false
1689 1483 Acked Write 4 Sent Done Done false
1690 1484 Acked Read 6 Sent Done Done false
1691 1485 Acked Write 2 Sent Done Done false
1692 1486 Acked Read 5 Sent Done Done false
1693 1487 Acked Flush 0 Sent Done Done false
1694 1488 Acked Write 3 Sent Done Done false
1695 1489 Acked Write 8 Sent Done Done false
1696 1490 Acked Read 10 Sent Done Done false
1697 1491 Acked Read 8 Sent Done Done false
1698 1492 Acked Flush 0 Sent Done Done false
1699 1493 Acked Read 1 Sent Done Done false
1700 1494 Acked Read 10 Sent Done Done false
1701 1495 Acked Read 4 Sent Done Done false
1702 1496 Acked Read 3 Sent Done Done false
1703 1497 Acked Read 3 Sent Done Done false
1704 1498 Acked Write 3 Sent Done Done false
1705 1499 Acked Read 4 Sent Done Done false
1706 1500 Acked Flush 0 Sent Done Done false
1707 1501 Acked Read 7 Sent Done Done false
1708 1502 Acked Read 7 Sent Done Done false
1709 1503 Acked Read 8 Sent Done Done false
1710 1504 Acked Write 2 Sent Done Done false
1711 1505 Acked Flush 0 Sent Done Done false
1712 1506 Acked Read 1 Sent Done Done false
1713 1507 Acked Write 5 Sent Done Done false
1714 1508 Acked Flush 0 Sent Done Done false
1715 1509 Acked Read 2 Sent Done Done false
1716 1510 Acked Write 10 Sent Done Done false
1717 1511 Acked Write 2 Sent Done Done false
1718 1512 Acked Write 6 Sent Done Done false
1719 1513 Acked Write 1 Sent Done Done false
1720 1514 Acked Read 4 Sent Done Done false
1721 1515 Acked Read 6 Sent Done Done false
1722 1516 Acked Read 1 Sent Done Done false
1723 1517 Acked Read 7 Sent Done Done false
1724 1518 Acked Write 7 Sent Done Done false
1725 1519 Acked Read 5 Sent Done Done false
1726 1520 Acked Flush 0 Sent Done Done false
1727 1521 Acked Write 8 Sent Done Done false
1728 1522 Acked Read 5 Sent Done Done false
1729 1523 Acked Write 8 Sent Done Done false
1730 1524 Acked Read 2 Sent Done Done false
1731 1525 Acked Flush 0 Sent Done Done false
1732 1526 Acked Read 8 Sent Done Done false
1733 1527 Acked Read 5 Sent Done Done false
1734 1528 Acked Read 3 Sent Done Done false
1735 1529 Acked Write 2 Sent Done Done false
1736 1530 Acked Read 5 Sent Done Done false
1737 1531 Acked Read 3 Sent Done Done false
1738 1532 Acked Write 1 Sent Done Done false
1739 1533 Acked Read 9 Sent Done Done false
1740 1534 Acked Read 2 Sent Done Done false
1741 1535 Acked Flush 0 Sent Done Done false
1742 1536 Acked Read 4 Sent Done Done false
1743 1537 Acked Read 5 Sent Done Done false
1744 1538 Acked Read 5 Sent Done Done false
1745 1539 Acked Read 4 Sent Done Done false
1746 1540 Acked Write 1 Sent Done Done false
1747 1541 Acked Read 9 Sent Done Done false
1748 1542 Acked Write 10 Sent Done Done false
1749 1543 Acked Read 5 Sent Done Done false
1750 1544 Acked Write 10 Sent Done Done false
1751 1545 Acked Write 9 Sent Done Done false
1752 1546 Acked Write 9 Sent Done Done false
1753 1547 Acked Read 8 Sent Done Done false
1754 1548 Acked Write 8 Sent Done Done false
1755 1549 Acked Read 4 Sent Done Done false
1756 1550 Acked Write 6 Sent Done Done false
1757 1551 Acked Read 3 Sent Done Done false
1758 1552 Acked Read 9 Sent Done Done false
1759 1553 Acked Write 1 Sent Done Done false
1760 1554 Acked Read 10 Sent Done Done false
1761 1555 Acked Read 3 Sent Done Done false
1762 1556 Acked Flush 0 Sent Done Done false
1763 1557 Acked Read 10 Sent Done Done false
1764 1558 Acked Read 4 Sent Done Done false
1765 1559 Acked Read 3 Sent Done Done false
1766 1560 Acked Write 1 Sent Done Done false
1767 1561 Acked Read 5 Sent Done Done false
1768 1562 Acked Read 6 Sent Done Done false
1769 1563 Acked Write 6 Sent Done Done false
1770 1564 Acked Read 2 Sent Done Done false
1771 1565 Acked Write 10 Sent Done Done false
1772 1566 Acked Flush 0 Sent Done Done false
1773 1567 Acked Write 2 Sent Done Done false
1774 1568 Acked Write 8 Sent Done Done false
1775 1569 Acked Write 10 Sent Done Done false
1776 1570 Acked Write 5 Sent Done Done false
1777 1571 Acked Write 5 Sent Done Done false
1778 1572 Acked Read 6 Sent Done Done false
1779 1573 Acked Read 8 Sent Done Done false
1780 1574 Acked Write 3 Sent Done Done false
1781 1575 Acked Write 3 Sent Done Done false
1782 1576 Acked Read 6 Sent Done Done false
1783 1577 Acked Read 9 Sent Done Done false
1784 1578 Acked Write 5 Sent Done Done false
1785 1579 Acked Read 7 Sent Done Done false
1786 1580 Acked Read 3 Sent Done Done false
1787 1581 Acked Write 2 Sent Done Done false
1788 1582 Acked Read 6 Sent Done Done false
1789 1583 Acked Read 9 Sent Done Done false
1790 1584 Acked Write 1 Sent Done Done false
1791 1585 Acked Read 7 Sent Done Done false
1792 1586 Acked Flush 0 Sent Done Done false
1793 1587 Acked Write 6 Sent Done Done false
1794 1588 Acked Read 5 Sent Done Done false
1795 1589 Acked Read 1 Sent Done Done false
1796 1590 Acked Write 9 Sent Done Done false
1797 1591 Acked Read 4 Sent Done Done false
1798 1592 Acked Read 3 Sent Done Done false
1799 1593 Acked Write 9 Sent Done Done false
1800 1594 Acked Write 4 Sent Done Done false
1801 1595 Acked Flush 0 Sent Done Done false
1802 1596 Acked Read 2 Sent Done Done false
1803 1597 Acked Write 8 Sent Done Done false
1804 1598 Acked Write 9 Sent Done Done false
1805 1599 Acked Write 2 Sent Done Done false
1806 1600 Acked Write 3 Sent Done Done false
1807 1601 Acked Write 4 Sent Done Done false
1808 1602 Acked Read 5 Sent Done Done false
1809 1603 Acked Write 8 Sent Done Done false
1810 STATES DS:0 DS:1 DS:2 TOTAL
1811 Sent 302 0 0 302
1812 Done 0 302 302 604
1813 Skipped 0 0 0 0
1814 Error 0 0 0 0
1815 Last Flush: 1301 1595 1595
1816 Downstairs last five completed: 1301 1300 1299 1298 1297
1817 Upstairs last five completed: 1603 1602 1601 1600 1599
1818 CLIENT: Up:0 ds:302 act:2
1819 ----------------------------------------------------------------
1820 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
1821 States: Offline Active Active
1822 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1823 1302 Acked Write 6 Sent Done Done false
1824 1303 Acked Flush 0 Sent Done Done false
1825 1304 Acked Read 9 Sent Done Done false
1826 1305 Acked Read 5 Sent Done Done false
1827 1306 Acked Read 3 Sent Done Done false
1828 1307 Acked Read 3 Sent Done Done false
1829 1308 Acked Write 2 Sent Done Done false
1830 1309 Acked Read 4 Sent Done Done false
1831 1310 Acked Write 5 Sent Done Done false
1832 1311 Acked Write 5 Sent Done Done false
1833 1312 Acked Write 6 Sent Done Done false
1834 1313 Acked Read 6 Sent Done Done false
1835 1314 Acked Write 2 Sent Done Done false
1836 1315 Acked Write 6 Sent Done Done false
1837 1316 Acked Read 1 Sent Done Done false
1838 1317 Acked Flush 0 Sent Done Done false
1839 1318 Acked Flush 0 Sent Done Done false
1840 1319 Acked Read 6 Sent Done Done false
1841 1320 Acked Write 7 Sent Done Done false
1842 1321 Acked Read 8 Sent Done Done false
1843 1322 Acked Write 2 Sent Done Done false
1844 1323 Acked Flush 0 Sent Done Done false
1845 1324 Acked Read 3 Sent Done Done false
1846 1325 Acked Read 5 Sent Done Done false
1847 1326 Acked Write 6 Sent Done Done false
1848 1327 Acked Read 5 Sent Done Done false
1849 1328 Acked Read 1 Sent Done Done false
1850 1329 Acked Read 3 Sent Done Done false
1851 1330 Acked Write 3 Sent Done Done false
1852 1331 Acked Flush 0 Sent Done Done false
1853 1332 Acked Read 1 Sent Done Done false
1854 1333 Acked Flush 0 Sent Done Done false
1855 1334 Acked Read 6 Sent Done Done false
1856 1335 Acked Read 4 Sent Done Done false
1857 1336 Acked Write 2 Sent Done Done false
1858 1337 Acked Read 3 Sent Done Done false
1859 1338 Acked Read 6 Sent Done Done false
1860 1339 Acked Read 8 Sent Done Done false
1861 1340 Acked Read 6 Sent Done Done false
1862 1341 Acked Read 1 Sent Done Done false
1863 1342 Acked Read 4 Sent Done Done false
1864 1343 Acked Write 1 Sent Done Done false
1865 1344 Acked Read 2 Sent Done Done false
1866 1345 Acked Write 4 Sent Done Done false
1867 1346 Acked Read 8 Sent Done Done false
1868 1347 Acked Read 4 Sent Done Done false
1869 1348 Acked Write 5 Sent Done Done false
1870 1349 Acked Read 1 Sent Done Done false
1871 1350 Acked Read 1 Sent Done Done false
1872 1351 Acked Read 4 Sent Done Done false
1873 1352 Acked Write 5 Sent Done Done false
1874 1353 Acked Write 10 Sent Done Done false
1875 1354 Acked Read 3 Sent Done Done false
1876 1355 Acked Read 4 Sent Done Done false
1877 1356 Acked Flush 0 Sent Done Done false
1878 1357 Acked Read 1 Sent Done Done false
1879 1358 Acked Read 7 Sent Done Done false
1880 1359 Acked Read 10 Sent Done Done false
1881 1360 Acked Read 2 Sent Done Done false
1882 1361 Acked Read 2 Sent Done Done false
1883 1362 Acked Read 4 Sent Done Done false
1884 1363 Acked Write 1 Sent Done Done false
1885 1364 Acked Write 3 Sent Done Done false
1886 1365 Acked Read 10 Sent Done Done false
1887 1366 Acked Read 4 Sent Done Done false
1888 1367 Acked Write 10 Sent Done Done false
1889 1368 Acked Flush 0 Sent Done Done false
1890 1369 Acked Read 7 Sent Done Done false
1891 1370 Acked Read 3 Sent Done Done false
1892 1371 Acked Read 7 Sent Done Done false
1893 1372 Acked Read 2 Sent Done Done false
1894 1373 Acked Write 5 Sent Done Done false
1895 1374 Acked Write 1 Sent Done Done false
1896 1375 Acked Read 9 Sent Done Done false
1897 1376 Acked Read 3 Sent Done Done false
1898 1377 Acked Write 9 Sent Done Done false
1899 1378 Acked Write 7 Sent Done Done false
1900 1379 Acked Write 9 Sent Done Done false
1901 1380 Acked Read 9 Sent Done Done false
1902 1381 Acked Read 8 Sent Done Done false
1903 1382 Acked Read 6 Sent Done Done false
1904 1383 Acked Read 6 Sent Done Done false
1905 1384 Acked Read 3 Sent Done Done false
1906 1385 Acked Write 10 Sent Done Done false
1907 1386 Acked Read 3 Sent Done Done false
1908 1387 Acked Write 9 Sent Done Done false
1909 1388 Acked Write 2 Sent Done Done false
1910 1389 Acked Read 9 Sent Done Done false
1911 1390 Acked Read 10 Sent Done Done false
1912 1391 Acked Write 5 Sent Done Done false
1913 1392 Acked Flush 0 Sent Done Done false
1914 1393 Acked Read 10 Sent Done Done false
1915 1394 Acked Write 10 Sent Done Done false
1916 1395 Acked Write 8 Sent Done Done false
1917 1396 Acked Read 1 Sent Done Done false
1918 1397 Acked Read 9 Sent Done Done false
1919 1398 Acked Write 3 Sent Done Done false
1920 1399 Acked Flush 0 Sent Done Done false
1921 1400 Acked Write 7 Sent Done Done false
1922 1401 Acked Read 6 Sent Done Done false
1923 1402 Acked Flush 0 Sent Done Done false
1924 1403 Acked Flush 0 Sent Done Done false
1925 1404 Acked Write 5 Sent Done Done false
1926 1405 Acked Write 2 Sent Done Done false
1927 1406 Acked Read 10 Sent Done Done false
1928 1407 Acked Write 3 Sent Done Done false
1929 1408 Acked Read 9 Sent Done Done false
1930 1409 Acked Read 1 Sent Done Done false
1931 1410 Acked Write 2 Sent Done Done false
1932 1411 Acked Read 3 Sent Done Done false
1933 1412 Acked Read 2 Sent Done Done false
1934 1413 Acked Read 6 Sent Done Done false
1935 1414 Acked Write 6 Sent Done Done false
1936 1415 Acked Write 6 Sent Done Done false
1937 1416 Acked Read 7 Sent Done Done false
1938 1417 Acked Flush 0 Sent Done Done false
1939 1418 Acked Write 7 Sent Done Done false
1940 1419 Acked Read 1 Sent Done Done false
1941 1420 Acked Write 8 Sent Done Done false
1942 1421 Acked Read 3 Sent Done Done false
1943 1422 Acked Read 6 Sent Done Done false
1944 1423 Acked Read 4 Sent Done Done false
1945 1424 Acked Write 4 Sent Done Done false
1946 1425 Acked Read 5 Sent Done Done false
1947 1426 Acked Flush 0 Sent Done Done false
1948 1427 Acked Read 7 Sent Done Done false
1949 1428 Acked Read 10 Sent Done Done false
1950 1429 Acked Write 6 Sent Done Done false
1951 1430 Acked Write 9 Sent Done Done false
1952 1431 Acked Read 5 Sent Done Done false
1953 1432 Acked Write 10 Sent Done Done false
1954 1433 Acked Write 6 Sent Done Done false
1955 1434 Acked Write 6 Sent Done Done false
1956 1435 Acked Write 5 Sent Done Done false
1957 1436 Acked Read 7 Sent Done Done false
1958 1437 Acked Write 2 Sent Done Done false
1959 1438 Acked Write 1 Sent Done Done false
1960 1439 Acked Read 3 Sent Done Done false
1961 1440 Acked Flush 0 Sent Done Done false
1962 1441 Acked Read 9 Sent Done Done false
1963 1442 Acked Read 6 Sent Done Done false
1964 1443 Acked Write 10 Sent Done Done false
1965 1444 Acked Read 9 Sent Done Done false
1966 1445 Acked Write 8 Sent Done Done false
1967 1446 Acked Read 6 Sent Done Done false
1968 1447 Acked Read 9 Sent Done Done false
1969 1448 Acked Read 8 Sent Done Done false
1970 1449 Acked Read 2 Sent Done Done false
1971 1450 Acked Read 4 Sent Done Done false
1972 1451 Acked Write 1 Sent Done Done false
1973 1452 Acked Flush 0 Sent Done Done false
1974 1453 Acked Write 3 Sent Done Done false
1975 1454 Acked Read 9 Sent Done Done false
1976 1455 Acked Read 7 Sent Done Done false
1977 1456 Acked Read 1 Sent Done Done false
1978 1457 Acked Write 3 Sent Done Done false
1979 1458 Acked Read 4 Sent Done Done false
1980 1459 Acked Read 1 Sent Done Done false
1981 1460 Acked Flush 0 Sent Done Done false
1982 1461 Acked Read 6 Sent Done Done false
1983 1462 Acked Read 10 Sent Done Done false
1984 1463 Acked Write 3 Sent Done Done false
1985 1464 Acked Write 6 Sent Done Done false
1986 1465 Acked Write 7 Sent Done Done false
1987 1466 Acked Read 3 Sent Done Done false
1988 1467 Acked Write 7 Sent Done Done false
1989 1468 Acked Read 9 Sent Done Done false
1990 1469 Acked Read 2 Sent Done Done false
1991 1470 Acked Read 2 Sent Done Done false
1992 1471 Acked Flush 0 Sent Done Done false
1993 1472 Acked Read 5 Sent Done Done false
1994 1473 Acked Read 1 Sent Done Done false
1995 1474 Acked Read 10 Sent Done Done false
1996 1475 Acked Read 7 Sent Done Done false
1997 1476 Acked Write 8 Sent Done Done false
1998 1477 Acked Read 1 Sent Done Done false
1999 1478 Acked Write 4 Sent Done Done false
2000 1479 Acked Read 1 Sent Done Done false
2001 1480 Acked Flush 0 Sent Done Done false
2002 1481 Acked Flush 0 Sent Done Done false
2003 1482 Acked Write 3 Sent Done Done false
2004 1483 Acked Write 4 Sent Done Done false
2005 1484 Acked Read 6 Sent Done Done false
2006 1485 Acked Write 2 Sent Done Done false
2007 1486 Acked Read 5 Sent Done Done false
2008 1487 Acked Flush 0 Sent Done Done false
2009 1488 Acked Write 3 Sent Done Done false
2010 1489 Acked Write 8 Sent Done Done false
2011 1490 Acked Read 10 Sent Done Done false
2012 1491 Acked Read 8 Sent Done Done false
2013 1492 Acked Flush 0 Sent Done Done false
2014 1493 Acked Read 1 Sent Done Done false
2015 1494 Acked Read 10 Sent Done Done false
2016 1495 Acked Read 4 Sent Done Done false
2017 1496 Acked Read 3 Sent Done Done false
2018 1497 Acked Read 3 Sent Done Done false
2019 1498 Acked Write 3 Sent Done Done false
2020 1499 Acked Read 4 Sent Done Done false
2021 1500 Acked Flush 0 Sent Done Done false
2022 1501 Acked Read 7 Sent Done Done false
2023 1502 Acked Read 7 Sent Done Done false
2024 1503 Acked Read 8 Sent Done Done false
2025 1504 Acked Write 2 Sent Done Done false
2026 1505 Acked Flush 0 Sent Done Done false
2027 1506 Acked Read 1 Sent Done Done false
2028 1507 Acked Write 5 Sent Done Done false
2029 1508 Acked Flush 0 Sent Done Done false
2030 1509 Acked Read 2 Sent Done Done false
2031 1510 Acked Write 10 Sent Done Done false
2032 1511 Acked Write 2 Sent Done Done false
2033 1512 Acked Write 6 Sent Done Done false
2034 1513 Acked Write 1 Sent Done Done false
2035 1514 Acked Read 4 Sent Done Done false
2036 1515 Acked Read 6 Sent Done Done false
2037 1516 Acked Read 1 Sent Done Done false
2038 1517 Acked Read 7 Sent Done Done false
2039 1518 Acked Write 7 Sent Done Done false
2040 1519 Acked Read 5 Sent Done Done false
2041 1520 Acked Flush 0 Sent Done Done false
2042 1521 Acked Write 8 Sent Done Done false
2043 1522 Acked Read 5 Sent Done Done false
2044 1523 Acked Write 8 Sent Done Done false
2045 1524 Acked Read 2 Sent Done Done false
2046 1525 Acked Flush 0 Sent Done Done false
2047 1526 Acked Read 8 Sent Done Done false
2048 1527 Acked Read 5 Sent Done Done false
2049 1528 Acked Read 3 Sent Done Done false
2050 1529 Acked Write 2 Sent Done Done false
2051 1530 Acked Read 5 Sent Done Done false
2052 1531 Acked Read 3 Sent Done Done false
2053 1532 Acked Write 1 Sent Done Done false
2054 1533 Acked Read 9 Sent Done Done false
2055 1534 Acked Read 2 Sent Done Done false
2056 1535 Acked Flush 0 Sent Done Done false
2057 1536 Acked Read 4 Sent Done Done false
2058 1537 Acked Read 5 Sent Done Done false
2059 1538 Acked Read 5 Sent Done Done false
2060 1539 Acked Read 4 Sent Done Done false
2061 1540 Acked Write 1 Sent Done Done false
2062 1541 Acked Read 9 Sent Done Done false
2063 1542 Acked Write 10 Sent Done Done false
2064 1543 Acked Read 5 Sent Done Done false
2065 1544 Acked Write 10 Sent Done Done false
2066 1545 Acked Write 9 Sent Done Done false
2067 1546 Acked Write 9 Sent Done Done false
2068 1547 Acked Read 8 Sent Done Done false
2069 1548 Acked Write 8 Sent Done Done false
2070 1549 Acked Read 4 Sent Done Done false
2071 1550 Acked Write 6 Sent Done Done false
2072 1551 Acked Read 3 Sent Done Done false
2073 1552 Acked Read 9 Sent Done Done false
2074 1553 Acked Write 1 Sent Done Done false
2075 1554 Acked Read 10 Sent Done Done false
2076 1555 Acked Read 3 Sent Done Done false
2077 1556 Acked Flush 0 Sent Done Done false
2078 1557 Acked Read 10 Sent Done Done false
2079 1558 Acked Read 4 Sent Done Done false
2080 1559 Acked Read 3 Sent Done Done false
2081 1560 Acked Write 1 Sent Done Done false
2082 1561 Acked Read 5 Sent Done Done false
2083 1562 Acked Read 6 Sent Done Done false
2084 1563 Acked Write 6 Sent Done Done false
2085 1564 Acked Read 2 Sent Done Done false
2086 1565 Acked Write 10 Sent Done Done false
2087 1566 Acked Flush 0 Sent Done Done false
2088 1567 Acked Write 2 Sent Done Done false
2089 1568 Acked Write 8 Sent Done Done false
2090 1569 Acked Write 10 Sent Done Done false
2091 1570 Acked Write 5 Sent Done Done false
2092 1571 Acked Write 5 Sent Done Done false
2093 1572 Acked Read 6 Sent Done Done false
2094 1573 Acked Read 8 Sent Done Done false
2095 1574 Acked Write 3 Sent Done Done false
2096 1575 Acked Write 3 Sent Done Done false
2097 1576 Acked Read 6 Sent Done Done false
2098 1577 Acked Read 9 Sent Done Done false
2099 1578 Acked Write 5 Sent Done Done false
2100 1579 Acked Read 7 Sent Done Done false
2101 1580 Acked Read 3 Sent Done Done false
2102 1581 Acked Write 2 Sent Done Done false
2103 1582 Acked Read 6 Sent Done Done false
2104 1583 Acked Read 9 Sent Done Done false
2105 1584 Acked Write 1 Sent Done Done false
2106 1585 Acked Read 7 Sent Done Done false
2107 1586 Acked Flush 0 Sent Done Done false
2108 1587 Acked Write 6 Sent Done Done false
2109 1588 Acked Read 5 Sent Done Done false
2110 1589 Acked Read 1 Sent Done Done false
2111 1590 Acked Write 9 Sent Done Done false
2112 1591 Acked Read 4 Sent Done Done false
2113 1592 Acked Read 3 Sent Done Done false
2114 1593 Acked Write 9 Sent Done Done false
2115 1594 Acked Write 4 Sent Done Done false
2116 1595 Acked Flush 0 Sent Done Done false
2117 1596 Acked Read 2 Sent Done Done false
2118 1597 Acked Write 8 Sent Done Done false
2119 1598 Acked Write 9 Sent Done Done false
2120 1599 Acked Write 2 Sent Done Done false
2121 1600 Acked Write 3 Sent Done Done false
2122 1601 Acked Write 4 Sent Done Done false
2123 1602 Acked Read 5 Sent Done Done false
2124 1603 Acked Write 8 Sent Done Done false
2125 1604 Acked Flush 0 Sent Done Done false
2126 STATES DS:0 DS:1 DS:2 TOTAL
2127 Sent 303 0 0 303
2128 Done 0 303 303 606
2129 Skipped 0 0 0 0
2130 Error 0 0 0 0
2131 Last Flush: 1301 1604 1604
2132 Downstairs last five completed: 1301 1300 1299 1298 1297
2133 Upstairs last five completed: 1604 1603 1602 1601 1600
2134 CLIENT: Up:0 ds:303 act:2
21352024-11-07T14:03:44.078ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
21362024-11-07T14:03:44.078ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
21372024-11-07T14:03:44.078ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
21382024-11-07T14:03:44.078ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
21392024-11-07T14:03:44.078ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2140 [8810] got start action
2141 [8810] I am going to restart
2142 Starting downstairs at port 8810
2143 Make output file at "/tmp/dsc/downstairs-8810.txt"
2144 [8810][0] reports Starting
2145 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1201)
2146 [8810][0] reports Running
2147 ----------------------------------------------------------------
2148 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
2149 States: Offline Active Active
2150 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2151 1302 Acked Write 6 Sent Done Done false
2152 1303 Acked Flush 0 Sent Done Done false
2153 1304 Acked Read 9 Sent Done Done false
2154 1305 Acked Read 5 Sent Done Done false
2155 1306 Acked Read 3 Sent Done Done false
2156 1307 Acked Read 3 Sent Done Done false
2157 1308 Acked Write 2 Sent Done Done false
2158 1309 Acked Read 4 Sent Done Done false
2159 1310 Acked Write 5 Sent Done Done false
2160 1311 Acked Write 5 Sent Done Done false
2161 1312 Acked Write 6 Sent Done Done false
2162 1313 Acked Read 6 Sent Done Done false
2163 1314 Acked Write 2 Sent Done Done false
2164 1315 Acked Write 6 Sent Done Done false
2165 1316 Acked Read 1 Sent Done Done false
2166 1317 Acked Flush 0 Sent Done Done false
2167 1318 Acked Flush 0 Sent Done Done false
2168 1319 Acked Read 6 Sent Done Done false
2169 1320 Acked Write 7 Sent Done Done false
2170 1321 Acked Read 8 Sent Done Done false
2171 1322 Acked Write 2 Sent Done Done false
2172 1323 Acked Flush 0 Sent Done Done false
2173 1324 Acked Read 3 Sent Done Done false
2174 1325 Acked Read 5 Sent Done Done false
2175 1326 Acked Write 6 Sent Done Done false
2176 1327 Acked Read 5 Sent Done Done false
2177 1328 Acked Read 1 Sent Done Done false
2178 1329 Acked Read 3 Sent Done Done false
2179 1330 Acked Write 3 Sent Done Done false
2180 1331 Acked Flush 0 Sent Done Done false
2181 1332 Acked Read 1 Sent Done Done false
2182 1333 Acked Flush 0 Sent Done Done false
2183 1334 Acked Read 6 Sent Done Done false
2184 1335 Acked Read 4 Sent Done Done false
2185 1336 Acked Write 2 Sent Done Done false
2186 1337 Acked Read 3 Sent Done Done false
2187 1338 Acked Read 6 Sent Done Done false
2188 1339 Acked Read 8 Sent Done Done false
2189 1340 Acked Read 6 Sent Done Done false
2190 1341 Acked Read 1 Sent Done Done false
2191 1342 Acked Read 4 Sent Done Done false
2192 1343 Acked Write 1 Sent Done Done false
2193 1344 Acked Read 2 Sent Done Done false
2194 1345 Acked Write 4 Sent Done Done false
2195 1346 Acked Read 8 Sent Done Done false
2196 1347 Acked Read 4 Sent Done Done false
2197 1348 Acked Write 5 Sent Done Done false
2198 1349 Acked Read 1 Sent Done Done false
2199 1350 Acked Read 1 Sent Done Done false
2200 1351 Acked Read 4 Sent Done Done false
2201 1352 Acked Write 5 Sent Done Done false
2202 1353 Acked Write 10 Sent Done Done false
2203 1354 Acked Read 3 Sent Done Done false
2204 1355 Acked Read 4 Sent Done Done false
2205 1356 Acked Flush 0 Sent Done Done false
2206 1357 Acked Read 1 Sent Done Done false
2207 1358 Acked Read 7 Sent Done Done false
2208 1359 Acked Read 10 Sent Done Done false
2209 1360 Acked Read 2 Sent Done Done false
2210 1361 Acked Read 2 Sent Done Done false
2211 1362 Acked Read 4 Sent Done Done false
2212 1363 Acked Write 1 Sent Done Done false
2213 1364 Acked Write 3 Sent Done Done false
2214 1365 Acked Read 10 Sent Done Done false
2215 1366 Acked Read 4 Sent Done Done false
2216 1367 Acked Write 10 Sent Done Done false
2217 1368 Acked Flush 0 Sent Done Done false
2218 1369 Acked Read 7 Sent Done Done false
2219 1370 Acked Read 3 Sent Done Done false
2220 1371 Acked Read 7 Sent Done Done false
2221 1372 Acked Read 2 Sent Done Done false
2222 1373 Acked Write 5 Sent Done Done false
2223 1374 Acked Write 1 Sent Done Done false
2224 1375 Acked Read 9 Sent Done Done false
2225 1376 Acked Read 3 Sent Done Done false
2226 1377 Acked Write 9 Sent Done Done false
2227 1378 Acked Write 7 Sent Done Done false
2228 1379 Acked Write 9 Sent Done Done false
2229 1380 Acked Read 9 Sent Done Done false
2230 1381 Acked Read 8 Sent Done Done false
2231 1382 Acked Read 6 Sent Done Done false
2232 1383 Acked Read 6 Sent Done Done false
2233 1384 Acked Read 3 Sent Done Done false
2234 1385 Acked Write 10 Sent Done Done false
2235 1386 Acked Read 3 Sent Done Done false
2236 1387 Acked Write 9 Sent Done Done false
2237 1388 Acked Write 2 Sent Done Done false
2238 1389 Acked Read 9 Sent Done Done false
2239 1390 Acked Read 10 Sent Done Done false
2240 1391 Acked Write 5 Sent Done Done false
2241 1392 Acked Flush 0 Sent Done Done false
2242 1393 Acked Read 10 Sent Done Done false
2243 1394 Acked Write 10 Sent Done Done false
2244 1395 Acked Write 8 Sent Done Done false
2245 1396 Acked Read 1 Sent Done Done false
2246 1397 Acked Read 9 Sent Done Done false
2247 1398 Acked Write 3 Sent Done Done false
2248 1399 Acked Flush 0 Sent Done Done false
2249 1400 Acked Write 7 Sent Done Done false
2250 1401 Acked Read 6 Sent Done Done false
2251 1402 Acked Flush 0 Sent Done Done false
2252 1403 Acked Flush 0 Sent Done Done false
2253 1404 Acked Write 5 Sent Done Done false
2254 1405 Acked Write 2 Sent Done Done false
2255 1406 Acked Read 10 Sent Done Done false
2256 1407 Acked Write 3 Sent Done Done false
2257 1408 Acked Read 9 Sent Done Done false
2258 1409 Acked Read 1 Sent Done Done false
2259 1410 Acked Write 2 Sent Done Done false
2260 1411 Acked Read 3 Sent Done Done false
2261 1412 Acked Read 2 Sent Done Done false
2262 1413 Acked Read 6 Sent Done Done false
2263 1414 Acked Write 6 Sent Done Done false
2264 1415 Acked Write 6 Sent Done Done false
2265 1416 Acked Read 7 Sent Done Done false
2266 1417 Acked Flush 0 Sent Done Done false
2267 1418 Acked Write 7 Sent Done Done false
2268 1419 Acked Read 1 Sent Done Done false
2269 1420 Acked Write 8 Sent Done Done false
2270 1421 Acked Read 3 Sent Done Done false
2271 1422 Acked Read 6 Sent Done Done false
2272 1423 Acked Read 4 Sent Done Done false
2273 1424 Acked Write 4 Sent Done Done false
2274 1425 Acked Read 5 Sent Done Done false
2275 1426 Acked Flush 0 Sent Done Done false
2276 1427 Acked Read 7 Sent Done Done false
2277 1428 Acked Read 10 Sent Done Done false
2278 1429 Acked Write 6 Sent Done Done false
2279 1430 Acked Write 9 Sent Done Done false
2280 1431 Acked Read 5 Sent Done Done false
2281 1432 Acked Write 10 Sent Done Done false
2282 1433 Acked Write 6 Sent Done Done false
2283 1434 Acked Write 6 Sent Done Done false
2284 1435 Acked Write 5 Sent Done Done false
2285 1436 Acked Read 7 Sent Done Done false
2286 1437 Acked Write 2 Sent Done Done false
2287 1438 Acked Write 1 Sent Done Done false
2288 1439 Acked Read 3 Sent Done Done false
2289 1440 Acked Flush 0 Sent Done Done false
2290 1441 Acked Read 9 Sent Done Done false
2291 1442 Acked Read 6 Sent Done Done false
2292 1443 Acked Write 10 Sent Done Done false
2293 1444 Acked Read 9 Sent Done Done false
2294 1445 Acked Write 8 Sent Done Done false
2295 1446 Acked Read 6 Sent Done Done false
2296 1447 Acked Read 9 Sent Done Done false
2297 1448 Acked Read 8 Sent Done Done false
2298 1449 Acked Read 2 Sent Done Done false
2299 1450 Acked Read 4 Sent Done Done false
2300 1451 Acked Write 1 Sent Done Done false
2301 1452 Acked Flush 0 Sent Done Done false
2302 1453 Acked Write 3 Sent Done Done false
2303 1454 Acked Read 9 Sent Done Done false
2304 1455 Acked Read 7 Sent Done Done false
2305 1456 Acked Read 1 Sent Done Done false
2306 1457 Acked Write 3 Sent Done Done false
2307 1458 Acked Read 4 Sent Done Done false
2308 1459 Acked Read 1 Sent Done Done false
2309 1460 Acked Flush 0 Sent Done Done false
2310 1461 Acked Read 6 Sent Done Done false
2311 1462 Acked Read 10 Sent Done Done false
2312 1463 Acked Write 3 Sent Done Done false
2313 1464 Acked Write 6 Sent Done Done false
2314 1465 Acked Write 7 Sent Done Done false
2315 1466 Acked Read 3 Sent Done Done false
2316 1467 Acked Write 7 Sent Done Done false
2317 1468 Acked Read 9 Sent Done Done false
2318 1469 Acked Read 2 Sent Done Done false
2319 1470 Acked Read 2 Sent Done Done false
2320 1471 Acked Flush 0 Sent Done Done false
2321 1472 Acked Read 5 Sent Done Done false
2322 1473 Acked Read 1 Sent Done Done false
2323 1474 Acked Read 10 Sent Done Done false
2324 1475 Acked Read 7 Sent Done Done false
2325 1476 Acked Write 8 Sent Done Done false
2326 1477 Acked Read 1 Sent Done Done false
2327 1478 Acked Write 4 Sent Done Done false
2328 1479 Acked Read 1 Sent Done Done false
2329 1480 Acked Flush 0 Sent Done Done false
2330 1481 Acked Flush 0 Sent Done Done false
2331 1482 Acked Write 3 Sent Done Done false
2332 1483 Acked Write 4 Sent Done Done false
2333 1484 Acked Read 6 Sent Done Done false
2334 1485 Acked Write 2 Sent Done Done false
2335 1486 Acked Read 5 Sent Done Done false
2336 1487 Acked Flush 0 Sent Done Done false
2337 1488 Acked Write 3 Sent Done Done false
2338 1489 Acked Write 8 Sent Done Done false
2339 1490 Acked Read 10 Sent Done Done false
2340 1491 Acked Read 8 Sent Done Done false
2341 1492 Acked Flush 0 Sent Done Done false
2342 1493 Acked Read 1 Sent Done Done false
2343 1494 Acked Read 10 Sent Done Done false
2344 1495 Acked Read 4 Sent Done Done false
2345 1496 Acked Read 3 Sent Done Done false
2346 1497 Acked Read 3 Sent Done Done false
2347 1498 Acked Write 3 Sent Done Done false
2348 1499 Acked Read 4 Sent Done Done false
2349 1500 Acked Flush 0 Sent Done Done false
2350 1501 Acked Read 7 Sent Done Done false
2351 1502 Acked Read 7 Sent Done Done false
2352 1503 Acked Read 8 Sent Done Done false
2353 1504 Acked Write 2 Sent Done Done false
2354 1505 Acked Flush 0 Sent Done Done false
2355 1506 Acked Read 1 Sent Done Done false
2356 1507 Acked Write 5 Sent Done Done false
2357 1508 Acked Flush 0 Sent Done Done false
2358 1509 Acked Read 2 Sent Done Done false
2359 1510 Acked Write 10 Sent Done Done false
2360 1511 Acked Write 2 Sent Done Done false
2361 1512 Acked Write 6 Sent Done Done false
2362 1513 Acked Write 1 Sent Done Done false
2363 1514 Acked Read 4 Sent Done Done false
2364 1515 Acked Read 6 Sent Done Done false
2365 1516 Acked Read 1 Sent Done Done false
2366 1517 Acked Read 7 Sent Done Done false
2367 1518 Acked Write 7 Sent Done Done false
2368 1519 Acked Read 5 Sent Done Done false
2369 1520 Acked Flush 0 Sent Done Done false
2370 1521 Acked Write 8 Sent Done Done false
2371 1522 Acked Read 5 Sent Done Done false
2372 1523 Acked Write 8 Sent Done Done false
2373 1524 Acked Read 2 Sent Done Done false
2374 1525 Acked Flush 0 Sent Done Done false
2375 1526 Acked Read 8 Sent Done Done false
2376 1527 Acked Read 5 Sent Done Done false
2377 1528 Acked Read 3 Sent Done Done false
2378 1529 Acked Write 2 Sent Done Done false
2379 1530 Acked Read 5 Sent Done Done false
2380 1531 Acked Read 3 Sent Done Done false
2381 1532 Acked Write 1 Sent Done Done false
2382 1533 Acked Read 9 Sent Done Done false
2383 1534 Acked Read 2 Sent Done Done false
2384 1535 Acked Flush 0 Sent Done Done false
2385 1536 Acked Read 4 Sent Done Done false
2386 1537 Acked Read 5 Sent Done Done false
2387 1538 Acked Read 5 Sent Done Done false
2388 1539 Acked Read 4 Sent Done Done false
2389 1540 Acked Write 1 Sent Done Done false
2390 1541 Acked Read 9 Sent Done Done false
2391 1542 Acked Write 10 Sent Done Done false
2392 1543 Acked Read 5 Sent Done Done false
2393 1544 Acked Write 10 Sent Done Done false
2394 1545 Acked Write 9 Sent Done Done false
2395 1546 Acked Write 9 Sent Done Done false
2396 1547 Acked Read 8 Sent Done Done false
2397 1548 Acked Write 8 Sent Done Done false
2398 1549 Acked Read 4 Sent Done Done false
2399 1550 Acked Write 6 Sent Done Done false
2400 1551 Acked Read 3 Sent Done Done false
2401 1552 Acked Read 9 Sent Done Done false
2402 1553 Acked Write 1 Sent Done Done false
2403 1554 Acked Read 10 Sent Done Done false
2404 1555 Acked Read 3 Sent Done Done false
2405 1556 Acked Flush 0 Sent Done Done false
2406 1557 Acked Read 10 Sent Done Done false
2407 1558 Acked Read 4 Sent Done Done false
2408 1559 Acked Read 3 Sent Done Done false
2409 1560 Acked Write 1 Sent Done Done false
2410 1561 Acked Read 5 Sent Done Done false
2411 1562 Acked Read 6 Sent Done Done false
2412 1563 Acked Write 6 Sent Done Done false
2413 1564 Acked Read 2 Sent Done Done false
2414 1565 Acked Write 10 Sent Done Done false
2415 1566 Acked Flush 0 Sent Done Done false
2416 1567 Acked Write 2 Sent Done Done false
2417 1568 Acked Write 8 Sent Done Done false
2418 1569 Acked Write 10 Sent Done Done false
2419 1570 Acked Write 5 Sent Done Done false
2420 1571 Acked Write 5 Sent Done Done false
2421 1572 Acked Read 6 Sent Done Done false
2422 1573 Acked Read 8 Sent Done Done false
2423 1574 Acked Write 3 Sent Done Done false
2424 1575 Acked Write 3 Sent Done Done false
2425 1576 Acked Read 6 Sent Done Done false
2426 1577 Acked Read 9 Sent Done Done false
2427 1578 Acked Write 5 Sent Done Done false
2428 1579 Acked Read 7 Sent Done Done false
2429 1580 Acked Read 3 Sent Done Done false
2430 1581 Acked Write 2 Sent Done Done false
2431 1582 Acked Read 6 Sent Done Done false
2432 1583 Acked Read 9 Sent Done Done false
2433 1584 Acked Write 1 Sent Done Done false
2434 1585 Acked Read 7 Sent Done Done false
2435 1586 Acked Flush 0 Sent Done Done false
2436 1587 Acked Write 6 Sent Done Done false
2437 1588 Acked Read 5 Sent Done Done false
2438 1589 Acked Read 1 Sent Done Done false
2439 1590 Acked Write 9 Sent Done Done false
2440 1591 Acked Read 4 Sent Done Done false
2441 1592 Acked Read 3 Sent Done Done false
2442 1593 Acked Write 9 Sent Done Done false
2443 1594 Acked Write 4 Sent Done Done false
2444 1595 Acked Flush 0 Sent Done Done false
2445 1596 Acked Read 2 Sent Done Done false
2446 1597 Acked Write 8 Sent Done Done false
2447 1598 Acked Write 9 Sent Done Done false
2448 1599 Acked Write 2 Sent Done Done false
2449 1600 Acked Write 3 Sent Done Done false
2450 1601 Acked Write 4 Sent Done Done false
2451 1602 Acked Read 5 Sent Done Done false
2452 1603 Acked Write 8 Sent Done Done false
2453 1604 Acked Flush 0 Sent Done Done false
2454 STATES DS:0 DS:1 DS:2 TOTAL
2455 Sent 303 0 0 303
2456 Done 0 303 303 606
2457 Skipped 0 0 0 0
2458 Error 0 0 0 0
2459 Last Flush: 1301 1604 1604
2460 Downstairs last five completed: 1301 1300 1299 1298 1297
2461 Upstairs last five completed: 1604 1603 1602 1601 1600
2462 CLIENT: Up:0 ds:303 act:2
2463 ----------------------------------------------------------------
2464 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
2465 States: Offline Active Active
2466 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2467 1302 Acked Write 6 Sent Done Done false
2468 1303 Acked Flush 0 Sent Done Done false
2469 1304 Acked Read 9 Sent Done Done false
2470 1305 Acked Read 5 Sent Done Done false
2471 1306 Acked Read 3 Sent Done Done false
2472 1307 Acked Read 3 Sent Done Done false
2473 1308 Acked Write 2 Sent Done Done false
2474 1309 Acked Read 4 Sent Done Done false
2475 1310 Acked Write 5 Sent Done Done false
2476 1311 Acked Write 5 Sent Done Done false
2477 1312 Acked Write 6 Sent Done Done false
2478 1313 Acked Read 6 Sent Done Done false
2479 1314 Acked Write 2 Sent Done Done false
2480 1315 Acked Write 6 Sent Done Done false
2481 1316 Acked Read 1 Sent Done Done false
2482 1317 Acked Flush 0 Sent Done Done false
2483 1318 Acked Flush 0 Sent Done Done false
2484 1319 Acked Read 6 Sent Done Done false
2485 1320 Acked Write 7 Sent Done Done false
2486 1321 Acked Read 8 Sent Done Done false
2487 1322 Acked Write 2 Sent Done Done false
2488 1323 Acked Flush 0 Sent Done Done false
2489 1324 Acked Read 3 Sent Done Done false
2490 1325 Acked Read 5 Sent Done Done false
2491 1326 Acked Write 6 Sent Done Done false
2492 1327 Acked Read 5 Sent Done Done false
2493 1328 Acked Read 1 Sent Done Done false
2494 1329 Acked Read 3 Sent Done Done false
2495 1330 Acked Write 3 Sent Done Done false
2496 1331 Acked Flush 0 Sent Done Done false
2497 1332 Acked Read 1 Sent Done Done false
2498 1333 Acked Flush 0 Sent Done Done false
2499 1334 Acked Read 6 Sent Done Done false
2500 1335 Acked Read 4 Sent Done Done false
2501 1336 Acked Write 2 Sent Done Done false
2502 1337 Acked Read 3 Sent Done Done false
2503 1338 Acked Read 6 Sent Done Done false
2504 1339 Acked Read 8 Sent Done Done false
2505 1340 Acked Read 6 Sent Done Done false
2506 1341 Acked Read 1 Sent Done Done false
2507 1342 Acked Read 4 Sent Done Done false
2508 1343 Acked Write 1 Sent Done Done false
2509 1344 Acked Read 2 Sent Done Done false
2510 1345 Acked Write 4 Sent Done Done false
2511 1346 Acked Read 8 Sent Done Done false
2512 1347 Acked Read 4 Sent Done Done false
2513 1348 Acked Write 5 Sent Done Done false
2514 1349 Acked Read 1 Sent Done Done false
2515 1350 Acked Read 1 Sent Done Done false
2516 1351 Acked Read 4 Sent Done Done false
2517 1352 Acked Write 5 Sent Done Done false
2518 1353 Acked Write 10 Sent Done Done false
2519 1354 Acked Read 3 Sent Done Done false
2520 1355 Acked Read 4 Sent Done Done false
2521 1356 Acked Flush 0 Sent Done Done false
2522 1357 Acked Read 1 Sent Done Done false
2523 1358 Acked Read 7 Sent Done Done false
2524 1359 Acked Read 10 Sent Done Done false
2525 1360 Acked Read 2 Sent Done Done false
2526 1361 Acked Read 2 Sent Done Done false
2527 1362 Acked Read 4 Sent Done Done false
2528 1363 Acked Write 1 Sent Done Done false
2529 1364 Acked Write 3 Sent Done Done false
2530 1365 Acked Read 10 Sent Done Done false
2531 1366 Acked Read 4 Sent Done Done false
2532 1367 Acked Write 10 Sent Done Done false
2533 1368 Acked Flush 0 Sent Done Done false
2534 1369 Acked Read 7 Sent Done Done false
2535 1370 Acked Read 3 Sent Done Done false
2536 1371 Acked Read 7 Sent Done Done false
2537 1372 Acked Read 2 Sent Done Done false
2538 1373 Acked Write 5 Sent Done Done false
2539 1374 Acked Write 1 Sent Done Done false
2540 1375 Acked Read 9 Sent Done Done false
2541 1376 Acked Read 3 Sent Done Done false
2542 1377 Acked Write 9 Sent Done Done false
2543 1378 Acked Write 7 Sent Done Done false
2544 1379 Acked Write 9 Sent Done Done false
2545 1380 Acked Read 9 Sent Done Done false
2546 1381 Acked Read 8 Sent Done Done false
2547 1382 Acked Read 6 Sent Done Done false
2548 1383 Acked Read 6 Sent Done Done false
2549 1384 Acked Read 3 Sent Done Done false
2550 1385 Acked Write 10 Sent Done Done false
2551 1386 Acked Read 3 Sent Done Done false
2552 1387 Acked Write 9 Sent Done Done false
2553 1388 Acked Write 2 Sent Done Done false
2554 1389 Acked Read 9 Sent Done Done false
2555 1390 Acked Read 10 Sent Done Done false
2556 1391 Acked Write 5 Sent Done Done false
2557 1392 Acked Flush 0 Sent Done Done false
2558 1393 Acked Read 10 Sent Done Done false
2559 1394 Acked Write 10 Sent Done Done false
2560 1395 Acked Write 8 Sent Done Done false
2561 1396 Acked Read 1 Sent Done Done false
2562 1397 Acked Read 9 Sent Done Done false
2563 1398 Acked Write 3 Sent Done Done false
2564 1399 Acked Flush 0 Sent Done Done false
2565 1400 Acked Write 7 Sent Done Done false
2566 1401 Acked Read 6 Sent Done Done false
2567 1402 Acked Flush 0 Sent Done Done false
2568 1403 Acked Flush 0 Sent Done Done false
2569 1404 Acked Write 5 Sent Done Done false
2570 1405 Acked Write 2 Sent Done Done false
2571 1406 Acked Read 10 Sent Done Done false
2572 1407 Acked Write 3 Sent Done Done false
2573 1408 Acked Read 9 Sent Done Done false
2574 1409 Acked Read 1 Sent Done Done false
2575 1410 Acked Write 2 Sent Done Done false
2576 1411 Acked Read 3 Sent Done Done false
2577 1412 Acked Read 2 Sent Done Done false
2578 1413 Acked Read 6 Sent Done Done false
2579 1414 Acked Write 6 Sent Done Done false
2580 1415 Acked Write 6 Sent Done Done false
2581 1416 Acked Read 7 Sent Done Done false
2582 1417 Acked Flush 0 Sent Done Done false
2583 1418 Acked Write 7 Sent Done Done false
2584 1419 Acked Read 1 Sent Done Done false
2585 1420 Acked Write 8 Sent Done Done false
2586 1421 Acked Read 3 Sent Done Done false
2587 1422 Acked Read 6 Sent Done Done false
2588 1423 Acked Read 4 Sent Done Done false
2589 1424 Acked Write 4 Sent Done Done false
2590 1425 Acked Read 5 Sent Done Done false
2591 1426 Acked Flush 0 Sent Done Done false
2592 1427 Acked Read 7 Sent Done Done false
2593 1428 Acked Read 10 Sent Done Done false
2594 1429 Acked Write 6 Sent Done Done false
2595 1430 Acked Write 9 Sent Done Done false
2596 1431 Acked Read 5 Sent Done Done false
2597 1432 Acked Write 10 Sent Done Done false
2598 1433 Acked Write 6 Sent Done Done false
2599 1434 Acked Write 6 Sent Done Done false
2600 1435 Acked Write 5 Sent Done Done false
2601 1436 Acked Read 7 Sent Done Done false
2602 1437 Acked Write 2 Sent Done Done false
2603 1438 Acked Write 1 Sent Done Done false
2604 1439 Acked Read 3 Sent Done Done false
2605 1440 Acked Flush 0 Sent Done Done false
2606 1441 Acked Read 9 Sent Done Done false
2607 1442 Acked Read 6 Sent Done Done false
2608 1443 Acked Write 10 Sent Done Done false
2609 1444 Acked Read 9 Sent Done Done false
2610 1445 Acked Write 8 Sent Done Done false
2611 1446 Acked Read 6 Sent Done Done false
2612 1447 Acked Read 9 Sent Done Done false
2613 1448 Acked Read 8 Sent Done Done false
2614 1449 Acked Read 2 Sent Done Done false
2615 1450 Acked Read 4 Sent Done Done false
2616 1451 Acked Write 1 Sent Done Done false
2617 1452 Acked Flush 0 Sent Done Done false
2618 1453 Acked Write 3 Sent Done Done false
2619 1454 Acked Read 9 Sent Done Done false
2620 1455 Acked Read 7 Sent Done Done false
2621 1456 Acked Read 1 Sent Done Done false
2622 1457 Acked Write 3 Sent Done Done false
2623 1458 Acked Read 4 Sent Done Done false
2624 1459 Acked Read 1 Sent Done Done false
2625 1460 Acked Flush 0 Sent Done Done false
2626 1461 Acked Read 6 Sent Done Done false
2627 1462 Acked Read 10 Sent Done Done false
2628 1463 Acked Write 3 Sent Done Done false
2629 1464 Acked Write 6 Sent Done Done false
2630 1465 Acked Write 7 Sent Done Done false
2631 1466 Acked Read 3 Sent Done Done false
2632 1467 Acked Write 7 Sent Done Done false
2633 1468 Acked Read 9 Sent Done Done false
2634 1469 Acked Read 2 Sent Done Done false
2635 1470 Acked Read 2 Sent Done Done false
2636 1471 Acked Flush 0 Sent Done Done false
2637 1472 Acked Read 5 Sent Done Done false
2638 1473 Acked Read 1 Sent Done Done false
2639 1474 Acked Read 10 Sent Done Done false
2640 1475 Acked Read 7 Sent Done Done false
2641 1476 Acked Write 8 Sent Done Done false
2642 1477 Acked Read 1 Sent Done Done false
2643 1478 Acked Write 4 Sent Done Done false
2644 1479 Acked Read 1 Sent Done Done false
2645 1480 Acked Flush 0 Sent Done Done false
2646 1481 Acked Flush 0 Sent Done Done false
2647 1482 Acked Write 3 Sent Done Done false
2648 1483 Acked Write 4 Sent Done Done false
2649 1484 Acked Read 6 Sent Done Done false
2650 1485 Acked Write 2 Sent Done Done false
2651 1486 Acked Read 5 Sent Done Done false
2652 1487 Acked Flush 0 Sent Done Done false
2653 1488 Acked Write 3 Sent Done Done false
2654 1489 Acked Write 8 Sent Done Done false
2655 1490 Acked Read 10 Sent Done Done false
2656 1491 Acked Read 8 Sent Done Done false
2657 1492 Acked Flush 0 Sent Done Done false
2658 1493 Acked Read 1 Sent Done Done false
2659 1494 Acked Read 10 Sent Done Done false
2660 1495 Acked Read 4 Sent Done Done false
2661 1496 Acked Read 3 Sent Done Done false
2662 1497 Acked Read 3 Sent Done Done false
2663 1498 Acked Write 3 Sent Done Done false
2664 1499 Acked Read 4 Sent Done Done false
2665 1500 Acked Flush 0 Sent Done Done false
2666 1501 Acked Read 7 Sent Done Done false
2667 1502 Acked Read 7 Sent Done Done false
2668 1503 Acked Read 8 Sent Done Done false
2669 1504 Acked Write 2 Sent Done Done false
2670 1505 Acked Flush 0 Sent Done Done false
2671 1506 Acked Read 1 Sent Done Done false
2672 1507 Acked Write 5 Sent Done Done false
2673 1508 Acked Flush 0 Sent Done Done false
2674 1509 Acked Read 2 Sent Done Done false
2675 1510 Acked Write 10 Sent Done Done false
2676 1511 Acked Write 2 Sent Done Done false
2677 1512 Acked Write 6 Sent Done Done false
2678 1513 Acked Write 1 Sent Done Done false
2679 1514 Acked Read 4 Sent Done Done false
2680 1515 Acked Read 6 Sent Done Done false
2681 1516 Acked Read 1 Sent Done Done false
2682 1517 Acked Read 7 Sent Done Done false
2683 1518 Acked Write 7 Sent Done Done false
2684 1519 Acked Read 5 Sent Done Done false
2685 1520 Acked Flush 0 Sent Done Done false
2686 1521 Acked Write 8 Sent Done Done false
2687 1522 Acked Read 5 Sent Done Done false
2688 1523 Acked Write 8 Sent Done Done false
2689 1524 Acked Read 2 Sent Done Done false
2690 1525 Acked Flush 0 Sent Done Done false
2691 1526 Acked Read 8 Sent Done Done false
2692 1527 Acked Read 5 Sent Done Done false
2693 1528 Acked Read 3 Sent Done Done false
2694 1529 Acked Write 2 Sent Done Done false
2695 1530 Acked Read 5 Sent Done Done false
2696 1531 Acked Read 3 Sent Done Done false
2697 1532 Acked Write 1 Sent Done Done false
2698 1533 Acked Read 9 Sent Done Done false
2699 1534 Acked Read 2 Sent Done Done false
2700 1535 Acked Flush 0 Sent Done Done false
2701 1536 Acked Read 4 Sent Done Done false
2702 1537 Acked Read 5 Sent Done Done false
2703 1538 Acked Read 5 Sent Done Done false
2704 1539 Acked Read 4 Sent Done Done false
2705 1540 Acked Write 1 Sent Done Done false
2706 1541 Acked Read 9 Sent Done Done false
2707 1542 Acked Write 10 Sent Done Done false
2708 1543 Acked Read 5 Sent Done Done false
2709 1544 Acked Write 10 Sent Done Done false
2710 1545 Acked Write 9 Sent Done Done false
2711 1546 Acked Write 9 Sent Done Done false
2712 1547 Acked Read 8 Sent Done Done false
2713 1548 Acked Write 8 Sent Done Done false
2714 1549 Acked Read 4 Sent Done Done false
2715 1550 Acked Write 6 Sent Done Done false
2716 1551 Acked Read 3 Sent Done Done false
2717 1552 Acked Read 9 Sent Done Done false
2718 1553 Acked Write 1 Sent Done Done false
2719 1554 Acked Read 10 Sent Done Done false
2720 1555 Acked Read 3 Sent Done Done false
2721 1556 Acked Flush 0 Sent Done Done false
2722 1557 Acked Read 10 Sent Done Done false
2723 1558 Acked Read 4 Sent Done Done false
2724 1559 Acked Read 3 Sent Done Done false
2725 1560 Acked Write 1 Sent Done Done false
2726 1561 Acked Read 5 Sent Done Done false
2727 1562 Acked Read 6 Sent Done Done false
2728 1563 Acked Write 6 Sent Done Done false
2729 1564 Acked Read 2 Sent Done Done false
2730 1565 Acked Write 10 Sent Done Done false
2731 1566 Acked Flush 0 Sent Done Done false
2732 1567 Acked Write 2 Sent Done Done false
2733 1568 Acked Write 8 Sent Done Done false
2734 1569 Acked Write 10 Sent Done Done false
2735 1570 Acked Write 5 Sent Done Done false
2736 1571 Acked Write 5 Sent Done Done false
2737 1572 Acked Read 6 Sent Done Done false
2738 1573 Acked Read 8 Sent Done Done false
2739 1574 Acked Write 3 Sent Done Done false
2740 1575 Acked Write 3 Sent Done Done false
2741 1576 Acked Read 6 Sent Done Done false
2742 1577 Acked Read 9 Sent Done Done false
2743 1578 Acked Write 5 Sent Done Done false
2744 1579 Acked Read 7 Sent Done Done false
2745 1580 Acked Read 3 Sent Done Done false
2746 1581 Acked Write 2 Sent Done Done false
2747 1582 Acked Read 6 Sent Done Done false
2748 1583 Acked Read 9 Sent Done Done false
2749 1584 Acked Write 1 Sent Done Done false
2750 1585 Acked Read 7 Sent Done Done false
2751 1586 Acked Flush 0 Sent Done Done false
2752 1587 Acked Write 6 Sent Done Done false
2753 1588 Acked Read 5 Sent Done Done false
2754 1589 Acked Read 1 Sent Done Done false
2755 1590 Acked Write 9 Sent Done Done false
2756 1591 Acked Read 4 Sent Done Done false
2757 1592 Acked Read 3 Sent Done Done false
2758 1593 Acked Write 9 Sent Done Done false
2759 1594 Acked Write 4 Sent Done Done false
2760 1595 Acked Flush 0 Sent Done Done false
2761 1596 Acked Read 2 Sent Done Done false
2762 1597 Acked Write 8 Sent Done Done false
2763 1598 Acked Write 9 Sent Done Done false
2764 1599 Acked Write 2 Sent Done Done false
2765 1600 Acked Write 3 Sent Done Done false
2766 1601 Acked Write 4 Sent Done Done false
2767 1602 Acked Read 5 Sent Done Done false
2768 1603 Acked Write 8 Sent Done Done false
2769 1604 Acked Flush 0 Sent Done Done false
2770 STATES DS:0 DS:1 DS:2 TOTAL
2771 Sent 303 0 0 303
2772 Done 0 303 303 606
2773 Skipped 0 0 0 0
2774 Error 0 0 0 0
2775 Last Flush: 1301 1604 1604
2776 Downstairs last five completed: 1301 1300 1299 1298 1297
2777 Upstairs last five completed: 1604 1603 1602 1601 1600
2778 CLIENT: Up:0 ds:303 act:2
27792024-11-07T14:03:54.080ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27802024-11-07T14:03:54.080ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45609) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27812024-11-07T14:03:54.081ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27822024-11-07T14:03:54.082ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27832024-11-07T14:03:54.082ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27842024-11-07T14:03:54.082ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27852024-11-07T14:03:54.082ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27862024-11-07T14:03:54.082ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27872024-11-07T14:03:54.082ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27882024-11-07T14:03:54.082ZINFOcrucible: [0] client re-new 303 jobs since flush 1301 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27892024-11-07T14:03:54.082ZINFOcrucible: [0] Replayed 303 jobs since flush: 1301 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2790 ----------------------------------------------------------------
2791 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2792 Downstairs last five completed: 1604 1603 1602 1601 1600
2793 Upstairs last five completed: 1604 1603 1602 1601 1600
2794 CLIENT: Up:0 ds:0 act:3
2795 Replay: All jobs finished
2796 Main task has work to do, go find it
2797 got dsc Stop(2)
2798 stop 2
2799 [8830] Got stop action so:false kr:false
2800 Nov 07 14:03:54.377 INFO request completed, latency_us: 345, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 6595493e-9f17-4c20-87c3-c6a6bcfeffde, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
28012024-11-07T14:03:54.378ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
28022024-11-07T14:03:54.378ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
28032024-11-07T14:03:54.378ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2804 Nov 07 14:03:54.378 INFO request completed, latency_us: 222, response_code: 200, uri: /state/cid/2, method: GET, req_id: 97ba2877-2b47-4fb2-a78d-3a23c18b4d49, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
28052024-11-07T14:03:54.378ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
28062024-11-07T14:03:54.378ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
28072024-11-07T14:03:54.378ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
2808 [8830] Exited with: ExitStatus(unix_wait_status(9))
2809 [8830][2] reports Exit
2810 Nov 07 14:03:58.380 INFO request completed, latency_us: 227, response_code: 200, uri: /state/cid/2, method: GET, req_id: cc7d92fe-fd7c-4047-b70a-247571f76ae7, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
2811 001/300 Flush
2812 002/300 Read block 250 len 16384
2813 003/300 Write block 4895 len 24576 data: 2 2 2 2 2 2
2814 004/300 Read block 2816 len 32768
2815 005/300 Write block 181 len 24576 data: 3 3 2 3 3 3
2816 006/300 Read block 827 len 12288
2817 007/300 Read block 846 len 12288
2818 008/300 Write block 3646 len 24576 data: 2 2 2 2 2 2
2819 009/300 Write block 2048 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2820 010/300 Read block 4645 len 16384
2821 011/300 Read block 3524 len 24576
2822 012/300 Write block 1919 len 16384 data: 2 2 2 2
2823 013/300 Write block 1014 len 28672 data: 2 2 2 2 2 2 2
2824 014/300 Read block 4739 len 40960
2825 015/300 Read block 4809 len 40960
2826 016/300 Write block 24 len 40960 data: 3 3 3 3 3 2 2 2 2 2
2827 017/300 Read block 4342 len 24576
2828 018/300 Write block 1927 len 28672 data: 2 2 2 2 2 2 2
2829 019/300 Read block 2901 len 24576
2830 020/300 Write block 3486 len 12288 data: 2 2 2
2831 021/300 Read block 3259 len 28672
2832 022/300 Read block 3222 len 4096
2833 023/300 Write block 2088 len 20480 data: 2 2 2 2 2
2834 024/300 Flush
2835 025/300 Read block 1281 len 24576
2836 026/300 Write block 4724 len 12288 data: 2 2 2
2837 027/300 Flush
2838 028/300 Write block 3240 len 36864 data: 2 2 2 2 2 2 2 2 3
2839 029/300 Read block 1143 len 16384
2840 030/300 Read block 3942 len 40960
2841 031/300 Read block 1565 len 36864
2842 032/300 Write block 275 len 4096 data: 2
2843 033/300 Read block 339 len 8192
2844 034/300 Read block 3134 len 16384
2845 035/300 Write block 1142 len 4096 data: 2
2846 036/300 Read block 2489 len 24576
2847 037/300 Write block 4012 len 4096 data: 3
2848 038/300 Read block 3401 len 32768
2849 039/300 Read block 1558 len 32768
2850 040/300 Read block 3336 len 40960
2851 041/300 Write block 2976 len 12288 data: 2 2 3
2852 042/300 Flush
2853 043/300 Read block 1695 len 20480
2854 044/300 Read block 4171 len 20480
2855 045/300 Read block 2412 len 40960
2856 046/300 Read block 2863 len 12288
2857 047/300 Write block 2421 len 40960 data: 2 2 3 3 3 3 3 3 2 2
2858 048/300 Flush
2859 049/300 Write block 2626 len 16384 data: 2 2 2 2
2860 050/300 Read block 2283 len 12288
2861 051/300 Write block 1325 len 32768 data: 3 3 3 3 3 3 2 2
2862 052/300 Read block 4276 len 28672
2863 053/300 Read block 4269 len 8192
2864 054/300 Write block 2539 len 40960 data: 3 3 3 3 3 3 2 2 2 2
2865 055/300 Flush
2866 056/300 Write block 3972 len 12288 data: 3 3 3
2867 057/300 Read block 3807 len 16384
2868 058/300 Flush
2869 059/300 Read block 4019 len 32768
2870 060/300 Flush
2871 061/300 Write block 4692 len 40960 data: 2 3 3 3 3 3 3 2 2 2
2872 062/300 Write block 4018 len 28672 data: 2 2 2 2 2 2 2
2873 063/300 Flush
2874 064/300 Read block 1333 len 8192
2875 065/300 Flush
2876 066/300 Write block 3635 len 4096 data: 2
2877 067/300 Read block 2710 len 4096
2878 068/300 Write block 854 len 8192 data: 2 2
2879 069/300 Write block 1627 len 8192 data: 2 2
2880 070/300 Read block 4400 len 4096
2881 071/300 Read block 3056 len 36864
2882 072/300 Read block 4189 len 20480
2883 073/300 Flush
2884 074/300 Write block 192 len 36864 data: 2 2 2 2 2 2 2 2 3
2885 075/300 Write block 3692 len 16384 data: 2 2 2 2
2886 076/300 Read block 3182 len 8192
2887 077/300 Read block 2905 len 36864
2888 078/300 Read block 3679 len 24576
2889 079/300 Read block 2444 len 28672
2890 080/300 Flush
2891 081/300 Write block 3686 len 4096 data: 2
2892 082/300 Write block 3130 len 8192 data: 2 2
2893 083/300 Flush
2894 084/300 Read block 405 len 4096
2895 085/300 Write block 4413 len 20480 data: 2 2 2 2 2
2896 086/300 Write block 3130 len 28672 data: 3 3 2 2 2 3 3
2897 087/300 Flush
2898 088/300 Read block 1827 len 12288
2899 089/300 Read block 2595 len 24576
2900 090/300 Flush
2901 091/300 Write block 4009 len 40960 data: 2 3 3 4 3 3 3 3 2 3
2902 092/300 Write block 2829 len 12288 data: 3 3 3
2903 093/300 Flush
2904 094/300 Read block 819 len 8192
2905 095/300 Write block 3833 len 20480 data: 3 3 3 3 2
2906 096/300 Write block 1505 len 20480 data: 2 2 2 2 2
2907 097/300 Write block 4864 len 4096 data: 2
2908 098/300 Read block 2674 len 20480
2909 099/300 Write block 857 len 4096 data: 2
2910 100/300 Write block 3911 len 28672 data: 2 2 2 2 2 2 2
2911 101/300 Read block 3455 len 28672
2912 102/300 Write block 4006 len 24576 data: 2 2 2 3 4 4
2913 103/300 Write block 1925 len 16384 data: 2 2 3 3
2914 104/300 Write block 2405 len 32768 data: 2 2 2 2 2 2 2 2
2915 105/300 Flush
2916 106/300 Write block 3903 len 40960 data: 3 2 2 2 2 2 2 2 3 3
2917 107/300 Write block 4549 len 8192 data: 3 3
2918 108/300 Write block 4495 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2919 109/300 Write block 3477 len 12288 data: 2 2 2
2920 110/300 Write block 914 len 20480 data: 3 3 3 3 3
2921 111/300 Read block 2228 len 12288
2922 112/300 Read block 2769 len 16384
2923 113/300 Read block 4781 len 24576
2924 114/300 Read block 3134 len 32768
2925 115/300 Write block 3660 len 32768 data: 2 2 2 2 2 2 2 2
2926 116/300 Read block 658 len 8192
2927 117/300 Write block 19 len 20480 data: 3 4 4 3 3
2928 118/300 Write block 99 len 8192 data: 2 2
2929 119/300 Write block 4911 len 40960 data: 3 3 2 2 2 2 2 2 2 2
2930 120/300 Write block 1085 len 20480 data: 2 2 2 2 3
2931 121/300 Read block 824 len 20480
2932 122/300 Flush
2933 123/300 Write block 4712 len 32768 data: 2 2 2 2 2 2 2 2
2934 124/300 Read block 2560 len 20480
2935 125/300 Write block 3731 len 8192 data: 2 2
2936 126/300 Read block 1904 len 4096
2937 127/300 Write block 2384 len 20480 data: 2 2 2 2 2
2938 128/300 Flush
2939 129/300 Read block 942 len 36864
2940 130/300 Write block 703 len 20480 data: 2 4 4 4 4
2941 131/300 Write block 2608 len 28672 data: 2 2 2 2 2 2 3
2942 132/300 Read block 2585 len 32768
2943 133/300 Flush
2944 134/300 Write block 4294 len 40960 data: 3 3 2 2 2 2 2 2 2 2
2945 135/300 Flush
2946 136/300 Write block 1449 len 8192 data: 4 4
2947 137/300 Flush
2948 138/300 Read block 2140 len 16384
2949 139/300 Read block 2820 len 24576
2950 140/300 Write block 2603 len 4096 data: 2
2951 141/300 Write block 3439 len 40960 data: 2 2 2 2 2 2 2 3 2 2
2952 142/300 Write block 2822 len 28672 data: 3 3 4 4 5 5 4
2953 143/300 Read block 1677 len 12288
2954 144/300 Read block 1884 len 36864
2955 145/300 Read block 381 len 40960
2956 146/300 Read block 2082 len 12288
2957 147/300 Write block 2059 len 24576 data: 2 2 2 2 2 3
2958 148/300 Read block 829 len 4096
2959 149/300 Write block 1954 len 32768 data: 2 2 2 2 2 2 2 2
2960 150/300 Write block 971 len 4096 data: 2
2961 151/300 Write block 249 len 12288 data: 2 2 2
2962 152/300 Write block 3581 len 24576 data: 2 2 2 2 2 2
2963 153/300 Write block 2411 len 8192 data: 3 3
2964 154/300 Write block 3424 len 40960 data: 3 3 3 3 3 3 3 3 2 2
2965 155/300 Read block 4639 len 28672
2966 156/300 Write block 2990 len 32768 data: 2 2 2 2 2 2 2 2
2967 157/300 Flush
2968 158/300 Write block 1437 len 28672 data: 2 2 3 3 3 5 5
2969 159/300 Flush
2970 160/300 Write block 789 len 8192 data: 2 2
2971 161/300 Read block 4233 len 12288
2972 162/300 Read block 1689 len 24576
2973 163/300 Write block 1356 len 36864 data: 2 2 2 2 2 2 2 2 2
2974 164/300 Write block 154 len 32768 data: 3 3 3 3 3 3 4 3
2975 165/300 Read block 1669 len 36864
2976 166/300 Read block 4153 len 4096
2977 167/300 Write block 4009 len 28672 data: 4 5 5 5 4 4 4
2978 168/300 Read block 4613 len 36864
2979 169/300 Read block 207 len 4096
2980 170/300 Read block 497 len 32768
2981 171/300 Flush
2982 172/300 Read block 3786 len 8192
2983 173/300 Read block 1989 len 20480
2984 174/300 Read block 4909 len 28672
2985 175/300 Read block 2133 len 32768
2986 176/300 Write block 4938 len 8192 data: 2 2
2987 177/300 Read block 1445 len 32768
2988 178/300 Read block 3892 len 28672
2989 179/300 Write block 4933 len 16384 data: 2 2 2 2
2990 180/300 Write block 1359 len 8192 data: 3 3
2991 181/300 Write block 3886 len 28672 data: 2 2 2 2 2 2 2
2992 182/300 Read block 2132 len 12288
2993 183/300 Read block 2707 len 8192
2994 184/300 Write block 1140 len 28672 data: 2 2 3 2 2 2 2
2995 185/300 Write block 3629 len 24576 data: 3 3 3 3 2 2
2996 186/300 Read block 1961 len 36864
2997 187/300 Flush
2998 188/300 Write block 54 len 36864 data: 3 3 2 2 2 2 2 2 2
2999 189/300 Write block 3480 len 36864 data: 2 2 2 3 3 2 3 3 3
3000 190/300 Read block 2238 len 36864
3001 191/300 Write block 518 len 20480 data: 2 2 2 2 2
3002 192/300 Read block 4093 len 8192
3003 193/300 Read block 3128 len 24576
3004 194/300 Read block 4434 len 16384
3005 195/300 Write block 179 len 16384 data: 2 3 4 4
3006 196/300 Read block 1038 len 8192
3007 197/300 Read block 4223 len 40960
3008 198/300 Read block 1175 len 20480
3009 199/300 Read block 350 len 8192
3010 200/300 Read block 4111 len 24576
3011 201/300 Read block 666 len 8192
3012 202/300 Read block 212 len 24576
3013 203/300 Read block 3940 len 40960
3014 204/300 Read block 710 len 4096
3015 205/300 Read block 303 len 28672
3016 206/300 Flush
3017 207/300 Read block 650 len 12288
3018 208/300 Read block 1273 len 40960
3019 209/300 Read block 4924 len 28672
3020 210/300 Flush
3021 211/300 Write block 2889 len 24576 data: 2 2 2 2 2 2
3022 212/300 Read block 4422 len 24576
3023 213/300 Write block 3400 len 20480 data: 3 3 3 3 3
3024 214/300 Write block 1050 len 16384 data: 2 2 2 2
3025 215/300 Read block 431 len 8192
3026 216/300 Read block 2263 len 36864
3027 217/300 Write block 3742 len 8192 data: 2 2
3028 218/300 Read block 3383 len 32768
3029 219/300 Read block 1440 len 16384
3030 220/300 Write block 3846 len 24576 data: 2 3 2 2 2 2
3031 221/300 Write block 307 len 8192 data: 2 2
3032 222/300 Write block 626 len 28672 data: 2 2 2 2 2 2 2
3033 223/300 Read block 630 len 32768
3034 224/300 Flush
3035 225/300 Read block 4124 len 28672
3036 226/300 Read block 426 len 20480
3037 227/300 Read block 135 len 24576
3038 228/300 Read block 3568 len 16384
3039 229/300 Write block 4115 len 20480 data: 2 2 2 2 3
3040 230/300 Write block 2741 len 32768 data: 2 2 2 2 2 2 2 2
3041 231/300 Read block 2624 len 40960
3042 232/300 Write block 2782 len 20480 data: 3 3 3 3 2
3043 233/300 Read block 952 len 24576
3044 234/300 Read block 4757 len 20480
3045 235/300 Read block 744 len 32768
3046 236/300 Read block 1792 len 40960
3047 237/300 Write block 3154 len 28672 data: 3 3 3 2 2 2 2
3048 238/300 Flush
3049 239/300 Read block 3788 len 20480
3050 240/300 Write block 1140 len 28672 data: 3 3 4 3 3 3 3
3051 241/300 Flush
3052 242/300 Read block 235 len 32768
3053 243/300 Read block 1290 len 20480
3054 244/300 Read block 4640 len 36864
3055 245/300 Read block 3926 len 40960
3056 246/300 Write block 4672 len 32768 data: 2 2 2 2 2 2 2 2
3057 247/300 Write block 543 len 24576 data: 3 2 2 2 3 3
3058 248/300 Flush
3059 249/300 Read block 1121 len 16384
3060 250/300 Read block 2187 len 36864
3061 251/300 Read block 4991 len 4096
3062 252/300 Read block 4387 len 8192
3063 253/300 Flush
3064 254/300 Read block 3433 len 4096
3065 255/300 Read block 1360 len 12288
3066 256/300 Write block 2561 len 28672 data: 2 2 2 2 2 2 2
3067 257/300 Write block 1188 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3068 258/300 Read block 4019 len 28672
3069 259/300 Write block 2517 len 20480 data: 2 2 2 2 2
3070 260/300 Write block 2026 len 16384 data: 2 2 2 2
3071 261/300 Write block 2882 len 28672 data: 2 2 2 2 2 2 2
3072 262/300 Read block 1713 len 24576
3073 263/300 Write block 3988 len 8192 data: 2 2
3074 264/300 Read block 1903 len 24576
3075 265/300 Read block 4886 len 8192
3076 266/300 Write block 4952 len 24576 data: 2 2 2 2 2 2
3077 267/300 Write block 4153 len 8192 data: 4 3
3078 268/300 Flush
3079 269/300 Flush
3080 270/300 Read block 4383 len 12288
3081 271/300 Write block 1924 len 36864 data: 2 3 3 4 4 3 3 3 3
3082 272/300 Read block 3641 len 24576
3083 273/300 Read block 1428 len 4096
3084 274/300 Read block 659 len 40960
3085 275/300 Read block 729 len 32768
3086 276/300 Flush
3087 277/300 Write block 3963 len 36864 data: 2 2 2 2 2 2 2 2 2
3088 278/300 Read block 3701 len 40960
3089 279/300 Read block 4009 len 20480
3090 280/300 Write block 3683 len 20480 data: 2 2 2 3 2
3091 281/300 Write block 1943 len 36864 data: 2 2 2 2 2 2 2 2 2
3092 282/300 Write block 3792 len 16384 data: 2 2 2 2
3093 283/300 Read block 4172 len 28672
3094 284/300 Read block 802 len 4096
3095 285/300 Write block 670 len 4096 data: 2
3096 286/300 Read block 411 len 40960
3097 287/300 Read block 461 len 32768
3098 288/300 Read block 2190 len 4096
3099 289/300 Read block 165 len 8192
3100 290/300 Write block 305 len 24576 data: 2 2 3 3 2 2
3101 291/300 Read block 1473 len 36864
3102 292/300 Write block 1940 len 28672 data: 2 2 2 3 3 3 3
3103 293/300 Read block 2261 len 40960
3104 294/300 Read block 2155 len 12288
3105 295/300 Read block 2703 len 40960
3106 296/300 Write block 3730 len 40960 data: 2 3 3 2 2 2 2 2 2 2
3107 297/300 Read block 3336 len 24576
3108 298/300 Read block 191 len 4096
3109 299/300 Read block 1593 len 28672
3110 300/300 Read block 3193 len 24576
3111 301/300 Write block 3991 len 4096 data: 2
3112 Main task has work to do, go find it
3113 got dsc Start(2)
3114 start 2
3115 Nov 07 14:03:58.676 INFO request completed, latency_us: 249, response_code: 204, uri: /start/cid/2, method: POST, req_id: 37825fcd-4c9a-4d9d-b9e7-b30312a5c0e5, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
3116 Replay: started 2, returned:Ok(())
3117 ----------------------------------------------------------------
3118 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3119 States: Active Active Offline
3120 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3121 1605 Acked Flush 0 Done Done Sent false
3122 1606 Acked Read 4 Done Done Sent false
3123 1607 Acked Write 6 Done Done Sent false
3124 1608 Acked Read 8 Done Done Sent false
3125 1609 Acked Write 6 Done Done Sent false
3126 1610 Acked Read 3 Done Done Sent false
3127 1611 Acked Read 3 Done Done Sent false
3128 1612 Acked Write 6 Done Done Sent false
3129 1613 Acked Write 10 Done Done Sent false
3130 1614 Acked Read 4 Done Done Sent false
3131 1615 Acked Read 6 Done Done Sent false
3132 1616 Acked Write 4 Done Done Sent false
3133 1617 Acked Write 7 Done Done Sent false
3134 1618 Acked Read 10 Done Done Sent false
3135 1619 Acked Read 10 Done Done Sent false
3136 1620 Acked Write 10 Done Done Sent false
3137 1621 Acked Read 6 Done Done Sent false
3138 1622 Acked Write 7 Done Done Sent false
3139 1623 Acked Read 6 Done Done Sent false
3140 1624 Acked Write 3 Done Done Sent false
3141 1625 Acked Read 7 Done Done Sent false
3142 1626 Acked Read 1 Done Done Sent false
3143 1627 Acked Write 5 Done Done Sent false
3144 1628 Acked Flush 0 Done Done Sent false
3145 1629 Acked Read 6 Done Done Sent false
3146 1630 Acked Write 3 Done Done Sent false
3147 1631 Acked Flush 0 Done Done Sent false
3148 1632 Acked Write 9 Done Done Sent false
3149 1633 Acked Read 4 Done Done Sent false
3150 1634 Acked Read 10 Done Done Sent false
3151 1635 Acked Read 9 Done Done Sent false
3152 1636 Acked Write 1 Done Done Sent false
3153 1637 Acked Read 2 Done Done Sent false
3154 1638 Acked Read 4 Done Done Sent false
3155 1639 Acked Write 1 Done Done Sent false
3156 1640 Acked Read 6 Done Done Sent false
3157 1641 Acked Write 1 Done Done Sent false
3158 1642 Acked Read 8 Done Done Sent false
3159 1643 Acked Read 8 Done Done Sent false
3160 1644 Acked Read 10 Done Done Sent false
3161 1645 Acked Write 3 Done Done Sent false
3162 1646 Acked Flush 0 Done Done Sent false
3163 1647 Acked Read 5 Done Done Sent false
3164 1648 Acked Read 5 Done Done Sent false
3165 1649 Acked Read 10 Done Done Sent false
3166 1650 Acked Read 3 Done Done Sent false
3167 1651 Acked Write 10 Done Done Sent false
3168 1652 Acked Flush 0 Done Done Sent false
3169 1653 Acked Write 4 Done Done Sent false
3170 1654 Acked Read 3 Done Done Sent false
3171 1655 Acked Write 8 Done Done Sent false
3172 1656 Acked Read 7 Done Done Sent false
3173 1657 Acked Read 2 Done Done Sent false
3174 1658 Acked Write 10 Done Done Sent false
3175 1659 Acked Flush 0 Done Done Sent false
3176 1660 Acked Write 3 Done Done Sent false
3177 1661 Acked Read 4 Done Done Sent false
3178 1662 Acked Flush 0 Done Done Sent false
3179 1663 Acked Read 8 Done Done Sent false
3180 1664 Acked Flush 0 Done Done Sent false
3181 1665 Acked Write 10 Done Done Sent false
3182 1666 Acked Write 7 Done Done Sent false
3183 1667 Acked Flush 0 Done Done Sent false
3184 1668 Acked Read 2 Done Done Sent false
3185 1669 Acked Flush 0 Done Done Sent false
3186 1670 Acked Write 1 Done Done Sent false
3187 1671 Acked Read 1 Done Done Sent false
3188 1672 Acked Write 2 Done Done Sent false
3189 1673 Acked Write 2 Done Done Sent false
3190 1674 Acked Read 1 Done Done Sent false
3191 1675 Acked Read 9 Done Done Sent false
3192 1676 Acked Read 5 Done Done Sent false
3193 1677 Acked Flush 0 Done Done Sent false
3194 1678 Acked Write 9 Done Done Sent false
3195 1679 Acked Write 4 Done Done Sent false
3196 1680 Acked Read 2 Done Done Sent false
3197 1681 Acked Read 9 Done Done Sent false
3198 1682 Acked Read 6 Done Done Sent false
3199 1683 Acked Read 7 Done Done Sent false
3200 1684 Acked Flush 0 Done Done Sent false
3201 1685 Acked Write 1 Done Done Sent false
3202 1686 Acked Write 2 Done Done Sent false
3203 1687 Acked Flush 0 Done Done Sent false
3204 1688 Acked Read 1 Done Done Sent false
3205 1689 Acked Write 5 Done Done Sent false
3206 1690 Acked Write 7 Done Done Sent false
3207 1691 Acked Flush 0 Done Done Sent false
3208 1692 Acked Read 3 Done Done Sent false
3209 1693 Acked Read 6 Done Done Sent false
3210 1694 Acked Flush 0 Done Done Sent false
3211 1695 Acked Write 10 Done Done Sent false
3212 1696 Acked Write 3 Done Done Sent false
3213 1697 Acked Flush 0 Done Done Sent false
3214 1698 Acked Read 2 Done Done Sent false
3215 1699 Acked Write 5 Done Done Sent false
3216 1700 Acked Write 5 Done Done Sent false
3217 1701 Acked Write 1 Done Done Sent false
3218 1702 Acked Read 5 Done Done Sent false
3219 1703 Acked Write 1 Done Done Sent false
3220 1704 Acked Write 7 Done Done Sent false
3221 1705 Acked Read 7 Done Done Sent false
3222 1706 Acked Write 6 Done Done Sent false
3223 1707 Acked Write 4 Done Done Sent false
3224 1708 Acked Write 8 Done Done Sent false
3225 1709 Acked Flush 0 Done Done Sent false
3226 1710 Acked Write 10 Done Done Sent false
3227 1711 Acked Write 2 Done Done Sent false
3228 1712 Acked Write 10 Done Done Sent false
3229 1713 Acked Write 3 Done Done Sent false
3230 1714 Acked Write 5 Done Done Sent false
3231 1715 Acked Read 3 Done Done Sent false
3232 1716 Acked Read 4 Done Done Sent false
3233 1717 Acked Read 6 Done Done Sent false
3234 1718 Acked Read 8 Done Done Sent false
3235 1719 Acked Write 8 Done Done Sent false
3236 1720 Acked Read 2 Done Done Sent false
3237 1721 Acked Write 5 Done Done Sent false
3238 1722 Acked Write 2 Done Done Sent false
3239 1723 Acked Write 10 Done Done Sent false
3240 1724 Acked Write 5 Done Done Sent false
3241 1725 Acked Read 5 Done Done Sent false
3242 1726 Acked Flush 0 Done Done Sent false
3243 1727 Acked Write 8 Done Done Sent false
3244 1728 Acked Read 5 Done Done Sent false
3245 1729 Acked Write 2 Done Done Sent false
3246 1730 Acked Read 1 Done Done Sent false
3247 1731 Acked Write 5 Done Done Sent false
3248 1732 Acked Flush 0 Done Done Sent false
3249 1733 Acked Read 9 Done Done Sent false
3250 1734 Acked Write 5 Done Done Sent false
3251 1735 Acked Write 7 Done Done Sent false
3252 1736 Acked Read 8 Done Done Sent false
3253 1737 Acked Flush 0 Done Done Sent false
3254 1738 Acked Write 10 Done Done Sent false
3255 1739 Acked Flush 0 Done Done Sent false
3256 1740 Acked Write 2 Done Done Sent false
3257 1741 Acked Flush 0 Done Done Sent false
3258 1742 Acked Read 4 Done Done Sent false
3259 1743 Acked Read 6 Done Done Sent false
3260 1744 Acked Write 1 Done Done Sent false
3261 1745 Acked Write 10 Done Done Sent false
3262 1746 Acked Write 7 Done Done Sent false
3263 1747 Acked Read 3 Done Done Sent false
3264 1748 Acked Read 9 Done Done Sent false
3265 1749 Acked Read 10 Done Done Sent false
3266 1750 Acked Read 3 Done Done Sent false
3267 1751 Acked Write 6 Done Done Sent false
3268 1752 Acked Read 1 Done Done Sent false
3269 1753 Acked Write 8 Done Done Sent false
3270 1754 Acked Write 1 Done Done Sent false
3271 1755 Acked Write 3 Done Done Sent false
3272 1756 Acked Write 6 Done Done Sent false
3273 1757 Acked Write 2 Done Done Sent false
3274 1758 Acked Write 10 Done Done Sent false
3275 1759 Acked Read 7 Done Done Sent false
3276 1760 Acked Write 8 Done Done Sent false
3277 1761 Acked Flush 0 Done Done Sent false
3278 1762 Acked Write 7 Done Done Sent false
3279 1763 Acked Flush 0 Done Done Sent false
3280 1764 Acked Write 2 Done Done Sent false
3281 1765 Acked Read 3 Done Done Sent false
3282 1766 Acked Read 6 Done Done Sent false
3283 1767 Acked Write 9 Done Done Sent false
3284 1768 Acked Write 8 Done Done Sent false
3285 1769 Acked Read 9 Done Done Sent false
3286 1770 Acked Read 1 Done Done Sent false
3287 1771 Acked Write 7 Done Done Sent false
3288 1772 Acked Read 9 Done Done Sent false
3289 1773 Acked Read 1 Done Done Sent false
3290 1774 Acked Read 8 Done Done Sent false
3291 1775 Acked Flush 0 Done Done Sent false
3292 1776 Acked Read 2 Done Done Sent false
3293 1777 Acked Read 5 Done Done Sent false
3294 1778 Acked Read 7 Done Done Sent false
3295 1779 Acked Read 8 Done Done Sent false
3296 1780 Acked Write 2 Done Done Sent false
3297 1781 Acked Read 8 Done Done Sent false
3298 1782 Acked Read 7 Done Done Sent false
3299 1783 Acked Write 4 Done Done Sent false
3300 1784 Acked Write 2 Done Done Sent false
3301 1785 Acked Write 7 Done Done Sent false
3302 1786 Acked Read 3 Done Done Sent false
3303 1787 Acked Read 2 Done Done Sent false
3304 1788 Acked Write 7 Done Done Sent false
3305 1789 Acked Write 6 Done Done Sent false
3306 1790 Acked Read 9 Done Done Sent false
3307 1791 Acked Flush 0 Done Done Sent false
3308 1792 Acked Write 9 Done Done Sent false
3309 1793 Acked Write 9 Done Done Sent false
3310 1794 Acked Read 9 Done Done Sent false
3311 1795 Acked Write 5 Done Done Sent false
3312 1796 Acked Read 2 Done Done Sent false
3313 1797 Acked Read 6 Done Done Sent false
3314 1798 Acked Read 4 Done Done Sent false
3315 1799 Acked Write 4 Done Done Sent false
3316 1800 Acked Read 2 Done Done Sent false
3317 1801 Acked Read 10 Done Done Sent false
3318 1802 Acked Read 5 Done Done Sent false
3319 1803 Acked Read 2 Done Done Sent false
3320 1804 Acked Read 6 Done Done Sent false
3321 1805 Acked Read 2 Done Done Sent false
3322 1806 Acked Read 6 Done Done Sent false
3323 1807 Acked Read 10 Done Done Sent false
3324 1808 Acked Read 1 Done Done Sent false
3325 1809 Acked Read 7 Done Done Sent false
3326 1810 Acked Flush 0 Done Done Sent false
3327 1811 Acked Read 3 Done Done Sent false
3328 1812 Acked Read 10 Done Done Sent false
3329 1813 Acked Read 7 Done Done Sent false
3330 1814 Acked Flush 0 Done Done Sent false
3331 1815 Acked Write 6 Done Done Sent false
3332 1816 Acked Read 6 Done Done Sent false
3333 1817 Acked Write 5 Done Done Sent false
3334 1818 Acked Write 4 Done Done Sent false
3335 1819 Acked Read 2 Done Done Sent false
3336 1820 Acked Read 9 Done Done Sent false
3337 1821 Acked Write 2 Done Done Sent false
3338 1822 Acked Read 8 Done Done Sent false
3339 1823 Acked Read 4 Done Done Sent false
3340 1824 Acked Write 6 Done Done Sent false
3341 1825 Acked Write 2 Done Done Sent false
3342 1826 Acked Write 7 Done Done Sent false
3343 1827 Acked Read 8 Done Done Sent false
3344 1828 Acked Flush 0 Done Done Sent false
3345 1829 Acked Read 7 Done Done Sent false
3346 1830 Acked Read 5 Done Done Sent false
3347 1831 Acked Read 6 Done Done Sent false
3348 1832 Acked Read 4 Done Done Sent false
3349 1833 Acked Write 5 Done Done Sent false
3350 1834 Acked Write 8 Done Done Sent false
3351 1835 Acked Read 10 Done Done Sent false
3352 1836 Acked Write 5 Done Done Sent false
3353 1837 Acked Read 6 Done Done Sent false
3354 1838 Acked Read 5 Done Done Sent false
3355 1839 Acked Read 8 Done Done Sent false
3356 1840 Acked Read 10 Done Done Sent false
3357 1841 Acked Write 7 Done Done Sent false
3358 1842 Acked Flush 0 Done Done Sent false
3359 1843 Acked Read 5 Done Done Sent false
3360 1844 Acked Write 7 Done Done Sent false
3361 1845 Acked Flush 0 Done Done Sent false
3362 1846 Acked Read 8 Done Done Sent false
3363 1847 Acked Read 5 Done Done Sent false
3364 1848 Acked Read 9 Done Done Sent false
3365 1849 Acked Read 10 Done Done Sent false
3366 1850 Acked Write 8 Done Done Sent false
3367 1851 Acked Write 6 Done Done Sent false
3368 1852 Acked Flush 0 Done Done Sent false
3369 1853 Acked Read 4 Done Done Sent false
3370 1854 Acked Read 9 Done Done Sent false
3371 1855 Acked Read 1 Done Done Sent false
3372 1856 Acked Read 2 Done Done Sent false
3373 1857 Acked Flush 0 Done Done Sent false
3374 1858 Acked Read 1 Done Done Sent false
3375 1859 Acked Read 3 Done Done Sent false
3376 1860 Acked Write 7 Done Done Sent false
3377 1861 Acked Write 10 Done Done Sent false
3378 1862 Acked Read 7 Done Done Sent false
3379 1863 Acked Write 5 Done Done Sent false
3380 1864 Acked Write 4 Done Done Sent false
3381 1865 Acked Write 7 Done Done Sent false
3382 1866 Acked Read 6 Done Done Sent false
3383 1867 Acked Write 2 Done Done Sent false
3384 1868 Acked Read 6 Done Done Sent false
3385 1869 Acked Read 2 Done Done Sent false
3386 1870 Acked Write 6 Done Done Sent false
3387 1871 Acked Write 2 Done Done Sent false
3388 1872 Acked Flush 0 Done Done Sent false
3389 1873 Acked Flush 0 Done Done Sent false
3390 1874 Acked Read 3 Done Done Sent false
3391 1875 Acked Write 9 Done Done Sent false
3392 1876 Acked Read 6 Done Done Sent false
3393 1877 Acked Read 1 Done Done Sent false
3394 1878 Acked Read 10 Done Done Sent false
3395 1879 Acked Read 8 Done Done Sent false
3396 1880 Acked Flush 0 Done Done Sent false
3397 1881 Acked Write 9 Done Done Sent false
3398 1882 Acked Read 10 Done Done Sent false
3399 1883 Acked Read 5 Done Done Sent false
3400 1884 Acked Write 5 Done Done Sent false
3401 1885 Acked Write 9 Done Done Sent false
3402 1886 Acked Write 4 Done Done Sent false
3403 1887 Acked Read 7 Done Done Sent false
3404 1888 Acked Read 1 Done Done Sent false
3405 1889 Acked Write 1 Done Done Sent false
3406 1890 Acked Read 10 Done Done Sent false
3407 1891 Acked Read 8 Done Done Sent false
3408 1892 Acked Read 1 Done Done Sent false
3409 1893 Acked Read 2 Done Done Sent false
3410 1894 Acked Write 6 Done Done Sent false
3411 1895 Acked Read 9 Done Done Sent false
3412 1896 Acked Write 7 Done Done Sent false
3413 1897 Acked Read 10 Done Done Sent false
3414 1898 Acked Read 3 Done Done Sent false
3415 1899 Acked Read 10 Done Done Sent false
3416 1900 Acked Write 10 Done Done Sent false
3417 1901 Acked Read 6 Done Done Sent false
3418 1902 Acked Read 1 Done Done Sent false
3419 1903 Acked Read 7 Done Done Sent false
3420 1904 Acked Read 6 Done Done Sent false
3421 1905 Acked Write 1 Done Done Sent false
3422 STATES DS:0 DS:1 DS:2 TOTAL
3423 Sent 0 0 301 301
3424 Done 301 301 0 602
3425 Skipped 0 0 0 0
3426 Error 0 0 0 0
3427 Last Flush: 1880 1880 1604
3428 Downstairs last five completed: 1604 1603 1602 1601 1600
3429 Upstairs last five completed: 1905 1904 1903 1902 1901
3430 CLIENT: Up:0 ds:301 act:2
3431 [8830] got start action
3432 [8830] I am going to restart
3433 Starting downstairs at port 8830
3434 Make output file at "/tmp/dsc/downstairs-8830.txt"
3435 [8830][2] reports Starting
3436 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1202)
3437 [8830][2] reports Running
3438 ----------------------------------------------------------------
3439 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3440 States: Active Active Offline
3441 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3442 1605 Acked Flush 0 Done Done Sent false
3443 1606 Acked Read 4 Done Done Sent false
3444 1607 Acked Write 6 Done Done Sent false
3445 1608 Acked Read 8 Done Done Sent false
3446 1609 Acked Write 6 Done Done Sent false
3447 1610 Acked Read 3 Done Done Sent false
3448 1611 Acked Read 3 Done Done Sent false
3449 1612 Acked Write 6 Done Done Sent false
3450 1613 Acked Write 10 Done Done Sent false
3451 1614 Acked Read 4 Done Done Sent false
3452 1615 Acked Read 6 Done Done Sent false
3453 1616 Acked Write 4 Done Done Sent false
3454 1617 Acked Write 7 Done Done Sent false
3455 1618 Acked Read 10 Done Done Sent false
3456 1619 Acked Read 10 Done Done Sent false
3457 1620 Acked Write 10 Done Done Sent false
3458 1621 Acked Read 6 Done Done Sent false
3459 1622 Acked Write 7 Done Done Sent false
3460 1623 Acked Read 6 Done Done Sent false
3461 1624 Acked Write 3 Done Done Sent false
3462 1625 Acked Read 7 Done Done Sent false
3463 1626 Acked Read 1 Done Done Sent false
3464 1627 Acked Write 5 Done Done Sent false
3465 1628 Acked Flush 0 Done Done Sent false
3466 1629 Acked Read 6 Done Done Sent false
3467 1630 Acked Write 3 Done Done Sent false
3468 1631 Acked Flush 0 Done Done Sent false
3469 1632 Acked Write 9 Done Done Sent false
3470 1633 Acked Read 4 Done Done Sent false
3471 1634 Acked Read 10 Done Done Sent false
3472 1635 Acked Read 9 Done Done Sent false
3473 1636 Acked Write 1 Done Done Sent false
3474 1637 Acked Read 2 Done Done Sent false
3475 1638 Acked Read 4 Done Done Sent false
3476 1639 Acked Write 1 Done Done Sent false
3477 1640 Acked Read 6 Done Done Sent false
3478 1641 Acked Write 1 Done Done Sent false
3479 1642 Acked Read 8 Done Done Sent false
3480 1643 Acked Read 8 Done Done Sent false
3481 1644 Acked Read 10 Done Done Sent false
3482 1645 Acked Write 3 Done Done Sent false
3483 1646 Acked Flush 0 Done Done Sent false
3484 1647 Acked Read 5 Done Done Sent false
3485 1648 Acked Read 5 Done Done Sent false
3486 1649 Acked Read 10 Done Done Sent false
3487 1650 Acked Read 3 Done Done Sent false
3488 1651 Acked Write 10 Done Done Sent false
3489 1652 Acked Flush 0 Done Done Sent false
3490 1653 Acked Write 4 Done Done Sent false
3491 1654 Acked Read 3 Done Done Sent false
3492 1655 Acked Write 8 Done Done Sent false
3493 1656 Acked Read 7 Done Done Sent false
3494 1657 Acked Read 2 Done Done Sent false
3495 1658 Acked Write 10 Done Done Sent false
3496 1659 Acked Flush 0 Done Done Sent false
3497 1660 Acked Write 3 Done Done Sent false
3498 1661 Acked Read 4 Done Done Sent false
3499 1662 Acked Flush 0 Done Done Sent false
3500 1663 Acked Read 8 Done Done Sent false
3501 1664 Acked Flush 0 Done Done Sent false
3502 1665 Acked Write 10 Done Done Sent false
3503 1666 Acked Write 7 Done Done Sent false
3504 1667 Acked Flush 0 Done Done Sent false
3505 1668 Acked Read 2 Done Done Sent false
3506 1669 Acked Flush 0 Done Done Sent false
3507 1670 Acked Write 1 Done Done Sent false
3508 1671 Acked Read 1 Done Done Sent false
3509 1672 Acked Write 2 Done Done Sent false
3510 1673 Acked Write 2 Done Done Sent false
3511 1674 Acked Read 1 Done Done Sent false
3512 1675 Acked Read 9 Done Done Sent false
3513 1676 Acked Read 5 Done Done Sent false
3514 1677 Acked Flush 0 Done Done Sent false
3515 1678 Acked Write 9 Done Done Sent false
3516 1679 Acked Write 4 Done Done Sent false
3517 1680 Acked Read 2 Done Done Sent false
3518 1681 Acked Read 9 Done Done Sent false
3519 1682 Acked Read 6 Done Done Sent false
3520 1683 Acked Read 7 Done Done Sent false
3521 1684 Acked Flush 0 Done Done Sent false
3522 1685 Acked Write 1 Done Done Sent false
3523 1686 Acked Write 2 Done Done Sent false
3524 1687 Acked Flush 0 Done Done Sent false
3525 1688 Acked Read 1 Done Done Sent false
3526 1689 Acked Write 5 Done Done Sent false
3527 1690 Acked Write 7 Done Done Sent false
3528 1691 Acked Flush 0 Done Done Sent false
3529 1692 Acked Read 3 Done Done Sent false
3530 1693 Acked Read 6 Done Done Sent false
3531 1694 Acked Flush 0 Done Done Sent false
3532 1695 Acked Write 10 Done Done Sent false
3533 1696 Acked Write 3 Done Done Sent false
3534 1697 Acked Flush 0 Done Done Sent false
3535 1698 Acked Read 2 Done Done Sent false
3536 1699 Acked Write 5 Done Done Sent false
3537 1700 Acked Write 5 Done Done Sent false
3538 1701 Acked Write 1 Done Done Sent false
3539 1702 Acked Read 5 Done Done Sent false
3540 1703 Acked Write 1 Done Done Sent false
3541 1704 Acked Write 7 Done Done Sent false
3542 1705 Acked Read 7 Done Done Sent false
3543 1706 Acked Write 6 Done Done Sent false
3544 1707 Acked Write 4 Done Done Sent false
3545 1708 Acked Write 8 Done Done Sent false
3546 1709 Acked Flush 0 Done Done Sent false
3547 1710 Acked Write 10 Done Done Sent false
3548 1711 Acked Write 2 Done Done Sent false
3549 1712 Acked Write 10 Done Done Sent false
3550 1713 Acked Write 3 Done Done Sent false
3551 1714 Acked Write 5 Done Done Sent false
3552 1715 Acked Read 3 Done Done Sent false
3553 1716 Acked Read 4 Done Done Sent false
3554 1717 Acked Read 6 Done Done Sent false
3555 1718 Acked Read 8 Done Done Sent false
3556 1719 Acked Write 8 Done Done Sent false
3557 1720 Acked Read 2 Done Done Sent false
3558 1721 Acked Write 5 Done Done Sent false
3559 1722 Acked Write 2 Done Done Sent false
3560 1723 Acked Write 10 Done Done Sent false
3561 1724 Acked Write 5 Done Done Sent false
3562 1725 Acked Read 5 Done Done Sent false
3563 1726 Acked Flush 0 Done Done Sent false
3564 1727 Acked Write 8 Done Done Sent false
3565 1728 Acked Read 5 Done Done Sent false
3566 1729 Acked Write 2 Done Done Sent false
3567 1730 Acked Read 1 Done Done Sent false
3568 1731 Acked Write 5 Done Done Sent false
3569 1732 Acked Flush 0 Done Done Sent false
3570 1733 Acked Read 9 Done Done Sent false
3571 1734 Acked Write 5 Done Done Sent false
3572 1735 Acked Write 7 Done Done Sent false
3573 1736 Acked Read 8 Done Done Sent false
3574 1737 Acked Flush 0 Done Done Sent false
3575 1738 Acked Write 10 Done Done Sent false
3576 1739 Acked Flush 0 Done Done Sent false
3577 1740 Acked Write 2 Done Done Sent false
3578 1741 Acked Flush 0 Done Done Sent false
3579 1742 Acked Read 4 Done Done Sent false
3580 1743 Acked Read 6 Done Done Sent false
3581 1744 Acked Write 1 Done Done Sent false
3582 1745 Acked Write 10 Done Done Sent false
3583 1746 Acked Write 7 Done Done Sent false
3584 1747 Acked Read 3 Done Done Sent false
3585 1748 Acked Read 9 Done Done Sent false
3586 1749 Acked Read 10 Done Done Sent false
3587 1750 Acked Read 3 Done Done Sent false
3588 1751 Acked Write 6 Done Done Sent false
3589 1752 Acked Read 1 Done Done Sent false
3590 1753 Acked Write 8 Done Done Sent false
3591 1754 Acked Write 1 Done Done Sent false
3592 1755 Acked Write 3 Done Done Sent false
3593 1756 Acked Write 6 Done Done Sent false
3594 1757 Acked Write 2 Done Done Sent false
3595 1758 Acked Write 10 Done Done Sent false
3596 1759 Acked Read 7 Done Done Sent false
3597 1760 Acked Write 8 Done Done Sent false
3598 1761 Acked Flush 0 Done Done Sent false
3599 1762 Acked Write 7 Done Done Sent false
3600 1763 Acked Flush 0 Done Done Sent false
3601 1764 Acked Write 2 Done Done Sent false
3602 1765 Acked Read 3 Done Done Sent false
3603 1766 Acked Read 6 Done Done Sent false
3604 1767 Acked Write 9 Done Done Sent false
3605 1768 Acked Write 8 Done Done Sent false
3606 1769 Acked Read 9 Done Done Sent false
3607 1770 Acked Read 1 Done Done Sent false
3608 1771 Acked Write 7 Done Done Sent false
3609 1772 Acked Read 9 Done Done Sent false
3610 1773 Acked Read 1 Done Done Sent false
3611 1774 Acked Read 8 Done Done Sent false
3612 1775 Acked Flush 0 Done Done Sent false
3613 1776 Acked Read 2 Done Done Sent false
3614 1777 Acked Read 5 Done Done Sent false
3615 1778 Acked Read 7 Done Done Sent false
3616 1779 Acked Read 8 Done Done Sent false
3617 1780 Acked Write 2 Done Done Sent false
3618 1781 Acked Read 8 Done Done Sent false
3619 1782 Acked Read 7 Done Done Sent false
3620 1783 Acked Write 4 Done Done Sent false
3621 1784 Acked Write 2 Done Done Sent false
3622 1785 Acked Write 7 Done Done Sent false
3623 1786 Acked Read 3 Done Done Sent false
3624 1787 Acked Read 2 Done Done Sent false
3625 1788 Acked Write 7 Done Done Sent false
3626 1789 Acked Write 6 Done Done Sent false
3627 1790 Acked Read 9 Done Done Sent false
3628 1791 Acked Flush 0 Done Done Sent false
3629 1792 Acked Write 9 Done Done Sent false
3630 1793 Acked Write 9 Done Done Sent false
3631 1794 Acked Read 9 Done Done Sent false
3632 1795 Acked Write 5 Done Done Sent false
3633 1796 Acked Read 2 Done Done Sent false
3634 1797 Acked Read 6 Done Done Sent false
3635 1798 Acked Read 4 Done Done Sent false
3636 1799 Acked Write 4 Done Done Sent false
3637 1800 Acked Read 2 Done Done Sent false
3638 1801 Acked Read 10 Done Done Sent false
3639 1802 Acked Read 5 Done Done Sent false
3640 1803 Acked Read 2 Done Done Sent false
3641 1804 Acked Read 6 Done Done Sent false
3642 1805 Acked Read 2 Done Done Sent false
3643 1806 Acked Read 6 Done Done Sent false
3644 1807 Acked Read 10 Done Done Sent false
3645 1808 Acked Read 1 Done Done Sent false
3646 1809 Acked Read 7 Done Done Sent false
3647 1810 Acked Flush 0 Done Done Sent false
3648 1811 Acked Read 3 Done Done Sent false
3649 1812 Acked Read 10 Done Done Sent false
3650 1813 Acked Read 7 Done Done Sent false
3651 1814 Acked Flush 0 Done Done Sent false
3652 1815 Acked Write 6 Done Done Sent false
3653 1816 Acked Read 6 Done Done Sent false
3654 1817 Acked Write 5 Done Done Sent false
3655 1818 Acked Write 4 Done Done Sent false
3656 1819 Acked Read 2 Done Done Sent false
3657 1820 Acked Read 9 Done Done Sent false
3658 1821 Acked Write 2 Done Done Sent false
3659 1822 Acked Read 8 Done Done Sent false
3660 1823 Acked Read 4 Done Done Sent false
3661 1824 Acked Write 6 Done Done Sent false
3662 1825 Acked Write 2 Done Done Sent false
3663 1826 Acked Write 7 Done Done Sent false
3664 1827 Acked Read 8 Done Done Sent false
3665 1828 Acked Flush 0 Done Done Sent false
3666 1829 Acked Read 7 Done Done Sent false
3667 1830 Acked Read 5 Done Done Sent false
3668 1831 Acked Read 6 Done Done Sent false
3669 1832 Acked Read 4 Done Done Sent false
3670 1833 Acked Write 5 Done Done Sent false
3671 1834 Acked Write 8 Done Done Sent false
3672 1835 Acked Read 10 Done Done Sent false
3673 1836 Acked Write 5 Done Done Sent false
3674 1837 Acked Read 6 Done Done Sent false
3675 1838 Acked Read 5 Done Done Sent false
3676 1839 Acked Read 8 Done Done Sent false
3677 1840 Acked Read 10 Done Done Sent false
3678 1841 Acked Write 7 Done Done Sent false
3679 1842 Acked Flush 0 Done Done Sent false
3680 1843 Acked Read 5 Done Done Sent false
3681 1844 Acked Write 7 Done Done Sent false
3682 1845 Acked Flush 0 Done Done Sent false
3683 1846 Acked Read 8 Done Done Sent false
3684 1847 Acked Read 5 Done Done Sent false
3685 1848 Acked Read 9 Done Done Sent false
3686 1849 Acked Read 10 Done Done Sent false
3687 1850 Acked Write 8 Done Done Sent false
3688 1851 Acked Write 6 Done Done Sent false
3689 1852 Acked Flush 0 Done Done Sent false
3690 1853 Acked Read 4 Done Done Sent false
3691 1854 Acked Read 9 Done Done Sent false
3692 1855 Acked Read 1 Done Done Sent false
3693 1856 Acked Read 2 Done Done Sent false
3694 1857 Acked Flush 0 Done Done Sent false
3695 1858 Acked Read 1 Done Done Sent false
3696 1859 Acked Read 3 Done Done Sent false
3697 1860 Acked Write 7 Done Done Sent false
3698 1861 Acked Write 10 Done Done Sent false
3699 1862 Acked Read 7 Done Done Sent false
3700 1863 Acked Write 5 Done Done Sent false
3701 1864 Acked Write 4 Done Done Sent false
3702 1865 Acked Write 7 Done Done Sent false
3703 1866 Acked Read 6 Done Done Sent false
3704 1867 Acked Write 2 Done Done Sent false
3705 1868 Acked Read 6 Done Done Sent false
3706 1869 Acked Read 2 Done Done Sent false
3707 1870 Acked Write 6 Done Done Sent false
3708 1871 Acked Write 2 Done Done Sent false
3709 1872 Acked Flush 0 Done Done Sent false
3710 1873 Acked Flush 0 Done Done Sent false
3711 1874 Acked Read 3 Done Done Sent false
3712 1875 Acked Write 9 Done Done Sent false
3713 1876 Acked Read 6 Done Done Sent false
3714 1877 Acked Read 1 Done Done Sent false
3715 1878 Acked Read 10 Done Done Sent false
3716 1879 Acked Read 8 Done Done Sent false
3717 1880 Acked Flush 0 Done Done Sent false
3718 1881 Acked Write 9 Done Done Sent false
3719 1882 Acked Read 10 Done Done Sent false
3720 1883 Acked Read 5 Done Done Sent false
3721 1884 Acked Write 5 Done Done Sent false
3722 1885 Acked Write 9 Done Done Sent false
3723 1886 Acked Write 4 Done Done Sent false
3724 1887 Acked Read 7 Done Done Sent false
3725 1888 Acked Read 1 Done Done Sent false
3726 1889 Acked Write 1 Done Done Sent false
3727 1890 Acked Read 10 Done Done Sent false
3728 1891 Acked Read 8 Done Done Sent false
3729 1892 Acked Read 1 Done Done Sent false
3730 1893 Acked Read 2 Done Done Sent false
3731 1894 Acked Write 6 Done Done Sent false
3732 1895 Acked Read 9 Done Done Sent false
3733 1896 Acked Write 7 Done Done Sent false
3734 1897 Acked Read 10 Done Done Sent false
3735 1898 Acked Read 3 Done Done Sent false
3736 1899 Acked Read 10 Done Done Sent false
3737 1900 Acked Write 10 Done Done Sent false
3738 1901 Acked Read 6 Done Done Sent false
3739 1902 Acked Read 1 Done Done Sent false
3740 1903 Acked Read 7 Done Done Sent false
3741 1904 Acked Read 6 Done Done Sent false
3742 1905 Acked Write 1 Done Done Sent false
3743 1906 Acked Flush 0 Done Done Sent false
3744 STATES DS:0 DS:1 DS:2 TOTAL
3745 Sent 0 0 302 302
3746 Done 302 302 0 604
3747 Skipped 0 0 0 0
3748 Error 0 0 0 0
3749 Last Flush: 1906 1906 1604
3750 Downstairs last five completed: 1604 1603 1602 1601 1600
3751 Upstairs last five completed: 1906 1905 1904 1903 1902
3752 CLIENT: Up:0 ds:302 act:2
37532024-11-07T14:04:04.379ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37542024-11-07T14:04:04.379ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64119) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37552024-11-07T14:04:04.380ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37562024-11-07T14:04:04.381ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37572024-11-07T14:04:04.381ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37582024-11-07T14:04:04.381ZINFOcrucible: send last flush ID to this DS: 1604 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37592024-11-07T14:04:04.381ZINFOcrucible: Replied this last flush ID: 1604 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37602024-11-07T14:04:04.381ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37612024-11-07T14:04:04.381ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37622024-11-07T14:04:04.381ZINFOcrucible: [2] client re-new 302 jobs since flush 1604 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37632024-11-07T14:04:04.381ZINFOcrucible: [2] Replayed 302 jobs since flush: 1604 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
3764 ----------------------------------------------------------------
3765 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3766 Downstairs last five completed: 1906 1905 1904 1903 1902
3767 Upstairs last five completed: 1906 1905 1904 1903 1902
3768 CLIENT: Up:0 ds:0 act:3
3769 Replay: All jobs finished
3770 Main task has work to do, go find it
3771 got dsc Stop(1)
3772 stop 1
3773 Nov 07 14:04:06.682 INFO request completed, latency_us: 382, response_code[8820] Got stop action so:false kr:false
3774 : 204, uri: /stop/cid/1, method: POST, req_id: daa21188-7735-4bb2-b108-e61b591093fa, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
37752024-11-07T14:04:06.683ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37762024-11-07T14:04:06.683ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37772024-11-07T14:04:06.683ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
3778 Nov 07 14:04:06.683 INFO request completed, latency_us: 143, response_code: 200, uri: /state/cid/1, method: GET, req_id: ee3cb299-674a-4836-bcec-024e61ebabe6, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
37792024-11-07T14:04:06.683ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37802024-11-07T14:04:06.683ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
37812024-11-07T14:04:06.683ZINFOcrucible: client task is exiting = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
3782 [8820] Exited with: ExitStatus(unix_wait_status(9))
3783 [8820][1] reports Exit
3784 Nov 07 14:04:10.685 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/1, method: GET, req_id: 6ab6cdd3-fb53-4090-8e8f-064db2fd1727, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
3785 001/300 Write block 1850 len 36864 data: 3 2 3 3 3 3 3 3 3
3786 002/300 Read block 4231 len 24576
3787 003/300 Read block 2833 len 28672
3788 004/300 Read block 1473 len 40960
3789 005/300 Flush
3790 006/300 Write block 1967 len 32768 data: 2 2 2 2 2 2 2 2
3791 007/300 Read block 264 len 36864
3792 008/300 Write block 1804 len 32768 data: 2 2 3 3 3 3 3 3
3793 009/300 Write block 4558 len 8192 data: 4 4
3794 010/300 Read block 3878 len 8192
3795 011/300 Write block 3720 len 12288 data: 2 2 2
3796 012/300 Write block 3285 len 24576 data: 2 2 2 2 2 2
3797 013/300 Write block 4771 len 12288 data: 3 3 3
3798 014/300 Read block 2403 len 12288
3799 015/300 Read block 3033 len 28672
3800 016/300 Read block 3441 len 32768
3801 017/300 Read block 1352 len 36864
3802 018/300 Read block 918 len 28672
3803 019/300 Read block 3275 len 24576
3804 020/300 Read block 4167 len 20480
3805 021/300 Read block 1421 len 4096
3806 022/300 Write block 2539 len 28672 data: 4 4 4 4 4 4 3
3807 023/300 Write block 2127 len 36864 data: 2 2 2 2 2 2 2 2 2
3808 024/300 Write block 1349 len 12288 data: 2 2 2
3809 025/300 Write block 2418 len 36864 data: 2 2 2 3 3 4 4 4 4
3810 026/300 Read block 490 len 40960
3811 027/300 Read block 3498 len 40960
3812 028/300 Read block 4126 len 8192
3813 029/300 Read block 3100 len 28672
3814 030/300 Write block 42 len 16384 data: 2 2 2 2
3815 031/300 Read block 2262 len 32768
3816 032/300 Read block 172 len 12288
3817 033/300 Write block 2475 len 12288 data: 2 2 2
3818 034/300 Flush
3819 035/300 Read block 3860 len 4096
3820 036/300 Write block 3627 len 16384 data: 3 3 4 4
3821 037/300 Write block 3552 len 8192 data: 2 2
3822 038/300 Flush
3823 039/300 Read block 2894 len 16384
3824 040/300 Read block 1444 len 28672
3825 041/300 Write block 4601 len 32768 data: 3 2 2 2 2 2 2 2
3826 042/300 Read block 3382 len 32768
3827 043/300 Write block 1329 len 20480 data: 4 4 3 3 2
3828 044/300 Write block 3232 len 20480 data: 3 3 3 3 3
3829 045/300 Read block 1940 len 16384
3830 046/300 Write block 2994 len 24576 data: 3 3 3 3 3 3
3831 047/300 Read block 2878 len 12288
3832 048/300 Write block 456 len 36864 data: 3 3 2 2 2 2 2 2 2
3833 049/300 Read block 1869 len 12288
3834 050/300 Read block 932 len 4096
3835 051/300 Write block 4473 len 24576 data: 2 2 2 2 2 2
3836 052/300 Write block 3747 len 32768 data: 2 2 2 2 2 2 2 2
3837 053/300 Read block 2515 len 16384
3838 054/300 Read block 905 len 32768
3839 055/300 Flush
3840 056/300 Read block 4760 len 4096
3841 057/300 Read block 428 len 8192
3842 058/300 Read block 4152 len 32768
3843 059/300 Read block 2416 len 36864
3844 060/300 Read block 4487 len 40960
3845 061/300 Read block 2328 len 40960
3846 062/300 Read block 317 len 4096
3847 063/300 Flush
3848 064/300 Read block 4182 len 4096
3849 065/300 Write block 236 len 12288 data: 2 2 2
3850 066/300 Read block 4995 len 20480
3851 067/300 Read block 1218 len 36864
3852 068/300 Write block 1595 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3853 069/300 Read block 2467 len 28672
3854 070/300 Read block 4567 len 36864
3855 071/300 Write block 3038 len 8192 data: 2 2
3856 072/300 Read block 721 len 24576
3857 073/300 Read block 4929 len 24576
3858 074/300 Read block 3051 len 40960
3859 075/300 Read block 2966 len 20480
3860 076/300 Write block 3194 len 8192 data: 2 2
3861 077/300 Write block 4336 len 28672 data: 2 2 2 2 2 2 2
3862 078/300 Write block 4597 len 32768 data: 3 3 4 4 4 3 3 3
3863 079/300 Write block 3895 len 28672 data: 3 3 3 3 3 3 3
3864 080/300 Write block 2365 len 24576 data: 2 2 2 2 2 2
3865 081/300 Write block 3058 len 36864 data: 2 2 2 2 2 2 2 2 2
3866 082/300 Read block 3834 len 32768
3867 083/300 Read block 3865 len 32768
3868 084/300 Read block 3675 len 20480
3869 085/300 Write block 817 len 36864 data: 2 2 2 2 2 2 2 2 2
3870 086/300 Flush
3871 087/300 Read block 2963 len 24576
3872 088/300 Read block 3998 len 24576
3873 089/300 Read block 4342 len 36864
3874 090/300 Read block 226 len 28672
3875 091/300 Read block 1731 len 4096
3876 092/300 Read block 3200 len 4096
3877 093/300 Flush
3878 094/300 Write block 2410 len 16384 data: 3 4 4 2
3879 095/300 Read block 2119 len 32768
3880 096/300 Read block 2148 len 36864
3881 097/300 Write block 438 len 32768 data: 3 3 3 3 3 3 3 2
3882 098/300 Read block 1239 len 40960
3883 099/300 Read block 2768 len 40960
3884 100/300 Read block 4003 len 12288
3885 101/300 Write block 3654 len 32768 data: 2 2 2 3 3 3 3 3
3886 102/300 Write block 2675 len 12288 data: 2 2 2
3887 103/300 Write block 821 len 24576 data: 3 3 3 3 3 2
3888 104/300 Write block 4951 len 40960 data: 2 3 3 3 3 3 3 2 2 2
3889 105/300 Write block 1553 len 36864 data: 2 2 2 2 2 2 2 2 2
3890 106/300 Write block 3681 len 32768 data: 2 2 3 3 3 4 3 2
3891 107/300 Read block 920 len 32768
3892 108/300 Read block 3128 len 20480
3893 109/300 Write block 394 len 12288 data: 2 3 3
3894 110/300 Read block 2493 len 16384
3895 111/300 Read block 640 len 40960
3896 112/300 Write block 3049 len 36864 data: 2 2 2 2 2 2 2 3 2
3897 113/300 Read block 4786 len 24576
3898 114/300 Write block 4331 len 16384 data: 2 2 2 2
3899 115/300 Read block 1504 len 20480
3900 116/300 Write block 751 len 16384 data: 3 3 2 2
3901 117/300 Write block 2979 len 8192 data: 2 2
3902 118/300 Flush
3903 119/300 Read block 4973 len 40960
3904 120/300 Write block 3751 len 24576 data: 3 3 3 3 2 2
3905 121/300 Read block 343 len 16384
3906 122/300 Read block 2756 len 16384
3907 123/300 Read block 3215 len 20480
3908 124/300 Write block 2969 len 36864 data: 2 2 2 2 2 2 2 3 3
3909 125/300 Flush
3910 126/300 Flush
3911 127/300 Read block 3541 len 4096
3912 128/300 Flush
3913 129/300 Write block 4288 len 24576 data: 2 2 2 2 3 3
3914 130/300 Write block 2161 len 40960 data: 2 2 3 3 3 3 3 3 3 3
3915 131/300 Write block 4075 len 24576 data: 2 2 2 2 2 2
3916 132/300 Write block 346 len 32768 data: 2 2 2 2 2 2 2 2
3917 133/300 Read block 4649 len 40960
3918 134/300 Read block 1237 len 32768
3919 135/300 Read block 1918 len 28672
3920 136/300 Write block 1722 len 20480 data: 2 2 2 2 2
3921 137/300 Write block 830 len 28672 data: 2 2 2 2 2 2 2
3922 138/300 Read block 2392 len 12288
3923 139/300 Write block 497 len 32768 data: 2 2 2 2 2 2 2 2
3924 140/300 Write block 4462 len 4096 data: 2
3925 141/300 Read block 3123 len 12288
3926 142/300 Read block 1632 len 8192
3927 143/300 Flush
3928 144/300 Write block 3735 len 28672 data: 3 3 3 3 3 2 2
3929 145/300 Write block 2874 len 4096 data: 2
3930 146/300 Write block 72 len 36864 data: 2 2 2 2 2 2 2 2 2
3931 147/300 Read block 4602 len 32768
3932 148/300 Read block 2431 len 32768
3933 149/300 Read block 888 len 8192
3934 150/300 Write block 4012 len 24576 data: 6 5 5 5 4 3
3935 151/300 Write block 4479 len 36864 data: 2 2 2 2 2 2 2 2 2
3936 152/300 Read block 2380 len 36864
3937 153/300 Write block 2268 len 32768 data: 2 2 2 2 2 2 2 2
3938 154/300 Write block 4490 len 12288 data: 2 2 2
3939 155/300 Write block 1597 len 36864 data: 3 3 3 3 3 3 3 3 2
3940 156/300 Write block 1385 len 36864 data: 2 2 2 2 2 2 2 2 2
3941 157/300 Write block 2648 len 16384 data: 2 2 2 2
3942 158/300 Write block 4095 len 8192 data: 2 2
3943 159/300 Write block 3724 len 12288 data: 2 2 2
3944 160/300 Read block 3918 len 20480
3945 161/300 Write block 3814 len 24576 data: 3 3 3 3 3 3
3946 162/300 Flush
3947 163/300 Read block 3573 len 40960
3948 164/300 Read block 3789 len 12288
3949 165/300 Read block 1932 len 36864
3950 166/300 Read block 511 len 16384
3951 167/300 Read block 1312 len 4096
3952 168/300 Write block 420 len 16384 data: 2 2 2 2
3953 169/300 Read block 1107 len 40960
3954 170/300 Read block 1018 len 20480
3955 171/300 Flush
3956 172/300 Write block 2528 len 20480 data: 2 2 2 2 2
3957 173/300 Write block 2226 len 32768 data: 2 2 2 2 2 2 2 2
3958 174/300 Read block 952 len 28672
3959 175/300 Write block 722 len 36864 data: 3 3 3 3 3 4 4 3 3
3960 176/300 Read block 246 len 32768
3961 177/300 Read block 4599 len 40960
3962 178/300 Read block 2357 len 20480
3963 179/300 Read block 626 len 12288
3964 180/300 Read block 3592 len 40960
3965 181/300 Flush
3966 182/300 Read block 2845 len 16384
3967 183/300 Read block 813 len 32768
3968 184/300 Read block 2079 len 36864
3969 185/300 Write block 381 len 12288 data: 2 2 2
3970 186/300 Read block 1245 len 24576
3971 187/300 Write block 894 len 24576 data: 3 3 3 2 2 2
3972 188/300 Write block 3132 len 24576 data: 3 3 3 4 4 2
3973 189/300 Write block 557 len 32768 data: 2 2 2 2 2 2 2 2
3974 190/300 Write block 4588 len 36864 data: 3 3 3 3 3 3 3 3 2
3975 191/300 Write block 1230 len 24576 data: 2 2 2 2 2 2
3976 192/300 Write block 4313 len 32768 data: 2 2 2 2 2 2 2 2
3977 193/300 Flush
3978 194/300 Write block 4330 len 36864 data: 2 3 3 3 3 2 3 3 3
3979 195/300 Write block 3509 len 16384 data: 2 2 2 2
3980 196/300 Read block 2187 len 12288
3981 197/300 Read block 2489 len 24576
3982 198/300 Write block 122 len 12288 data: 2 2 2
3983 199/300 Read block 1085 len 16384
3984 200/300 Read block 876 len 20480
3985 201/300 Read block 259 len 28672
3986 202/300 Write block 3595 len 40960 data: 2 2 2 2 2 2 2 2 2 3
3987 203/300 Write block 2521 len 40960 data: 3 2 2 2 2 2 2 3 3 3
3988 204/300 Read block 2923 len 28672
3989 205/300 Read block 3803 len 8192
3990 206/300 Write block 292 len 40960 data: 2 3 3 3 2 2 2 2 2 2
3991 207/300 Flush
3992 208/300 Write block 1541 len 32768 data: 2 2 2 2 2 2 2 2
3993 209/300 Read block 4386 len 24576
3994 210/300 Read block 361 len 20480
3995 211/300 Read block 3344 len 8192
3996 212/300 Write block 4687 len 4096 data: 3
3997 213/300 Read block 1243 len 40960
3998 214/300 Write block 2721 len 32768 data: 2 2 2 2 2 2 2 2
3999 215/300 Read block 2090 len 24576
4000 216/300 Read block 3932 len 32768
4001 217/300 Write block 632 len 20480 data: 3 2 2 2 2
4002 218/300 Read block 1990 len 12288
4003 219/300 Write block 2567 len 20480 data: 3 2 2 2 2
4004 220/300 Write block 4718 len 32768 data: 3 3 2 2 2 2 3 3
4005 221/300 Write block 3089 len 16384 data: 3 3 3 3
4006 222/300 Write block 4298 len 12288 data: 3 3 3
4007 223/300 Read block 1296 len 20480
4008 224/300 Read block 2115 len 12288
4009 225/300 Write block 2049 len 32768 data: 3 3 3 3 3 3 3 3
4010 226/300 Write block 380 len 20480 data: 2 3 3 3 2
4011 227/300 Read block 2192 len 16384
4012 228/300 Read block 1628 len 16384
4013 229/300 Write block 527 len 28672 data: 2 2 2 2 2 2 2
4014 230/300 Write block 3257 len 24576 data: 2 2 2 2 2 2
4015 231/300 Read block 2564 len 20480
4016 232/300 Read block 1834 len 36864
4017 233/300 Read block 4974 len 16384
4018 234/300 Write block 3328 len 32768 data: 3 3 2 2 2 2 2 3
4019 235/300 Read block 1773 len 32768
4020 236/300 Write block 402 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4021 237/300 Write block 2357 len 36864 data: 2 2 2 2 2 2 2 2 3
4022 238/300 Read block 2790 len 32768
4023 239/300 Write block 4608 len 32768 data: 3 2 2 2 2 2 2 2
4024 240/300 Read block 2908 len 24576
4025 241/300 Write block 4989 len 16384 data: 2 2 2 2
4026 242/300 Write block 438 len 12288 data: 4 4 4
4027 243/300 Write block 3882 len 12288 data: 3 3 2
4028 244/300 Read block 2130 len 28672
4029 245/300 Read block 3171 len 8192
4030 246/300 Read block 3636 len 4096
4031 247/300 Flush
4032 248/300 Read block 1569 len 28672
4033 249/300 Read block 4632 len 24576
4034 250/300 Read block 4602 len 40960
4035 251/300 Read block 1682 len 36864
4036 252/300 Read block 3382 len 8192
4037 253/300 Read block 3120 len 4096
4038 254/300 Read block 4836 len 20480
4039 255/300 Read block 2107 len 20480
4040 256/300 Read block 2554 len 4096
4041 257/300 Write block 485 len 16384 data: 3 3 3 3
4042 258/300 Read block 4421 len 20480
4043 259/300 Read block 4587 len 36864
4044 260/300 Read block 1587 len 8192
4045 261/300 Read block 2406 len 8192
4046 262/300 Read block 4058 len 28672
4047 263/300 Read block 2779 len 40960
4048 264/300 Read block 1805 len 40960
4049 265/300 Read block 4489 len 4096
4050 266/300 Write block 4965 len 28672 data: 2 2 2 2 2 2 2
4051 267/300 Write block 4980 len 28672 data: 3 2 2 2 2 2 2
4052 268/300 Flush
4053 269/300 Read block 1975 len 24576
4054 270/300 Write block 2394 len 32768 data: 2 2 2 2 2 2 2 2
4055 271/300 Write block 2181 len 32768 data: 2 2 2 2 2 2 2 2
4056 272/300 Read block 4477 len 20480
4057 273/300 Read block 3584 len 8192
4058 274/300 Write block 477 len 24576 data: 2 2 2 2 2 3
4059 275/300 Write block 1708 len 16384 data: 2 2 2 2
4060 276/300 Write block 3196 len 4096 data: 2
4061 277/300 Read block 1897 len 16384
4062 278/300 Read block 3193 len 12288
4063 279/300 Flush
4064 280/300 Read block 2445 len 40960
4065 281/300 Write block 678 len 32768 data: 2 2 2 2 2 3 3 2
4066 282/300 Write block 2565 len 20480 data: 3 3 4 3 3
4067 283/300 Read block 2542 len 24576
4068 284/300 Write block 1104 len 28672 data: 2 2 2 2 2 2 2
4069 285/300 Write block 2138 len 16384 data: 2 2 2 2
4070 286/300 Read block 1779 len 40960
4071 287/300 Write block 206 len 4096 data: 2
4072 288/300 Read block 291 len 32768
4073 289/300 Read block 1838 len 8192
4074 290/300 Read block 4214 len 40960
4075 291/300 Read block 3109 len 4096
4076 292/300 Write block 4269 len 20480 data: 3 3 3 3 3
4077 293/300 Write block 589 len 28672 data: 2 2 2 2 3 3 3
4078 294/300 Write block 2012 len 28672 data: 3 4 4 3 3 4 4
4079 295/300 Read block 76 len 20480
4080 296/300 Flush
4081 297/300 Write block 2017 len 40960 data: 5 5 4 4 3 3 2 2 2 3
4082 298/300 Read block 445 len 20480
4083 299/300 Read block 1102 len 32768
4084 300/300 Read block 3087 len 32768
4085 301/300 Read block 3531 len 24576
4086 Main task has work to do, go find it
4087 got dsc Start(1)
4088 start 1
4089 Nov 07 14:04:10.971 INFO request completed, latency_us: 200, response_code: 204, uri: /start/cid/1, method: POST, req_id: 11626aa1-f5e5-4c99-bbd2-d617d834f3a5, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
4090 Replay: started 1, returned:Ok(())
4091 ----------------------------------------------------------------
4092 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4093 States: Active Offline Active
4094 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4095 1907 Acked Write 9 Done Sent Done false
4096 1908 Acked Read 6 Done Sent Done false
4097 1909 Acked Read 7 Done Sent Done false
4098 1910 Acked Read 10 Done Sent Done false
4099 1911 Acked Flush 0 Done Sent Done false
4100 1912 Acked Write 8 Done Sent Done false
4101 1913 Acked Read 9 Done Sent Done false
4102 1914 Acked Write 8 Done Sent Done false
4103 1915 Acked Write 2 Done Sent Done false
4104 1916 Acked Read 2 Done Sent Done false
4105 1917 Acked Write 3 Done Sent Done false
4106 1918 Acked Write 6 Done Sent Done false
4107 1919 Acked Write 3 Done Sent Done false
4108 1920 Acked Read 3 Done Sent Done false
4109 1921 Acked Read 7 Done Sent Done false
4110 1922 Acked Read 8 Done Sent Done false
4111 1923 Acked Read 9 Done Sent Done false
4112 1924 Acked Read 7 Done Sent Done false
4113 1925 Acked Read 6 Done Sent Done false
4114 1926 Acked Read 5 Done Sent Done false
4115 1927 Acked Read 1 Done Sent Done false
4116 1928 Acked Write 7 Done Sent Done false
4117 1929 Acked Write 9 Done Sent Done false
4118 1930 Acked Write 3 Done Sent Done false
4119 1931 Acked Write 9 Done Sent Done false
4120 1932 Acked Read 10 Done Sent Done false
4121 1933 Acked Read 10 Done Sent Done false
4122 1934 Acked Read 2 Done Sent Done false
4123 1935 Acked Read 7 Done Sent Done false
4124 1936 Acked Write 4 Done Sent Done false
4125 1937 Acked Read 8 Done Sent Done false
4126 1938 Acked Read 3 Done Sent Done false
4127 1939 Acked Write 3 Done Sent Done false
4128 1940 Acked Flush 0 Done Sent Done false
4129 1941 Acked Read 1 Done Sent Done false
4130 1942 Acked Write 4 Done Sent Done false
4131 1943 Acked Write 2 Done Sent Done false
4132 1944 Acked Flush 0 Done Sent Done false
4133 1945 Acked Read 4 Done Sent Done false
4134 1946 Acked Read 7 Done Sent Done false
4135 1947 Acked Write 8 Done Sent Done false
4136 1948 Acked Read 8 Done Sent Done false
4137 1949 Acked Write 5 Done Sent Done false
4138 1950 Acked Write 5 Done Sent Done false
4139 1951 Acked Read 4 Done Sent Done false
4140 1952 Acked Write 6 Done Sent Done false
4141 1953 Acked Read 3 Done Sent Done false
4142 1954 Acked Write 9 Done Sent Done false
4143 1955 Acked Read 3 Done Sent Done false
4144 1956 Acked Read 1 Done Sent Done false
4145 1957 Acked Write 6 Done Sent Done false
4146 1958 Acked Write 8 Done Sent Done false
4147 1959 Acked Read 4 Done Sent Done false
4148 1960 Acked Read 8 Done Sent Done false
4149 1961 Acked Flush 0 Done Sent Done false
4150 1962 Acked Read 1 Done Sent Done false
4151 1963 Acked Read 2 Done Sent Done false
4152 1964 Acked Read 8 Done Sent Done false
4153 1965 Acked Read 9 Done Sent Done false
4154 1966 Acked Read 10 Done Sent Done false
4155 1967 Acked Read 10 Done Sent Done false
4156 1968 Acked Read 1 Done Sent Done false
4157 1969 Acked Flush 0 Done Sent Done false
4158 1970 Acked Read 1 Done Sent Done false
4159 1971 Acked Write 3 Done Sent Done false
4160 1972 Acked Read 5 Done Sent Done false
4161 1973 Acked Read 9 Done Sent Done false
4162 1974 Acked Write 10 Done Sent Done false
4163 1975 Acked Read 7 Done Sent Done false
4164 1976 Acked Read 9 Done Sent Done false
4165 1977 Acked Write 2 Done Sent Done false
4166 1978 Acked Read 6 Done Sent Done false
4167 1979 Acked Read 6 Done Sent Done false
4168 1980 Acked Read 10 Done Sent Done false
4169 1981 Acked Read 5 Done Sent Done false
4170 1982 Acked Write 2 Done Sent Done false
4171 1983 Acked Write 7 Done Sent Done false
4172 1984 Acked Write 8 Done Sent Done false
4173 1985 Acked Write 7 Done Sent Done false
4174 1986 Acked Write 6 Done Sent Done false
4175 1987 Acked Write 9 Done Sent Done false
4176 1988 Acked Read 8 Done Sent Done false
4177 1989 Acked Read 8 Done Sent Done false
4178 1990 Acked Read 5 Done Sent Done false
4179 1991 Acked Write 9 Done Sent Done false
4180 1992 Acked Flush 0 Done Sent Done false
4181 1993 Acked Read 6 Done Sent Done false
4182 1994 Acked Read 6 Done Sent Done false
4183 1995 Acked Read 9 Done Sent Done false
4184 1996 Acked Read 7 Done Sent Done false
4185 1997 Acked Read 1 Done Sent Done false
4186 1998 Acked Read 1 Done Sent Done false
4187 1999 Acked Flush 0 Done Sent Done false
4188 2000 Acked Write 4 Done Sent Done false
4189 2001 Acked Read 8 Done Sent Done false
4190 2002 Acked Read 9 Done Sent Done false
4191 2003 Acked Write 8 Done Sent Done false
4192 2004 Acked Read 10 Done Sent Done false
4193 2005 Acked Read 10 Done Sent Done false
4194 2006 Acked Read 3 Done Sent Done false
4195 2007 Acked Write 8 Done Sent Done false
4196 2008 Acked Write 3 Done Sent Done false
4197 2009 Acked Write 6 Done Sent Done false
4198 2010 Acked Write 10 Done Sent Done false
4199 2011 Acked Write 9 Done Sent Done false
4200 2012 Acked Write 8 Done Sent Done false
4201 2013 Acked Read 8 Done Sent Done false
4202 2014 Acked Read 5 Done Sent Done false
4203 2015 Acked Write 3 Done Sent Done false
4204 2016 Acked Read 4 Done Sent Done false
4205 2017 Acked Read 10 Done Sent Done false
4206 2018 Acked Write 9 Done Sent Done false
4207 2019 Acked Read 6 Done Sent Done false
4208 2020 Acked Write 4 Done Sent Done false
4209 2021 Acked Read 5 Done Sent Done false
4210 2022 Acked Write 4 Done Sent Done false
4211 2023 Acked Write 2 Done Sent Done false
4212 2024 Acked Flush 0 Done Sent Done false
4213 2025 Acked Read 10 Done Sent Done false
4214 2026 Acked Write 6 Done Sent Done false
4215 2027 Acked Read 4 Done Sent Done false
4216 2028 Acked Read 4 Done Sent Done false
4217 2029 Acked Read 5 Done Sent Done false
4218 2030 Acked Write 9 Done Sent Done false
4219 2031 Acked Flush 0 Done Sent Done false
4220 2032 Acked Flush 0 Done Sent Done false
4221 2033 Acked Read 1 Done Sent Done false
4222 2034 Acked Flush 0 Done Sent Done false
4223 2035 Acked Write 6 Done Sent Done false
4224 2036 Acked Write 10 Done Sent Done false
4225 2037 Acked Write 6 Done Sent Done false
4226 2038 Acked Write 8 Done Sent Done false
4227 2039 Acked Read 10 Done Sent Done false
4228 2040 Acked Read 8 Done Sent Done false
4229 2041 Acked Read 7 Done Sent Done false
4230 2042 Acked Write 5 Done Sent Done false
4231 2043 Acked Write 7 Done Sent Done false
4232 2044 Acked Read 3 Done Sent Done false
4233 2045 Acked Write 8 Done Sent Done false
4234 2046 Acked Write 1 Done Sent Done false
4235 2047 Acked Read 3 Done Sent Done false
4236 2048 Acked Read 2 Done Sent Done false
4237 2049 Acked Flush 0 Done Sent Done false
4238 2050 Acked Write 7 Done Sent Done false
4239 2051 Acked Write 1 Done Sent Done false
4240 2052 Acked Write 9 Done Sent Done false
4241 2053 Acked Read 8 Done Sent Done false
4242 2054 Acked Read 8 Done Sent Done false
4243 2055 Acked Read 2 Done Sent Done false
4244 2056 Acked Write 6 Done Sent Done false
4245 2057 Acked Write 9 Done Sent Done false
4246 2058 Acked Read 9 Done Sent Done false
4247 2059 Acked Write 8 Done Sent Done false
4248 2060 Acked Write 3 Done Sent Done false
4249 2061 Acked Write 9 Done Sent Done false
4250 2062 Acked Flush 0 Done Sent Done false
4251 2063 Acked Write 9 Done Sent Done false
4252 2064 Acked Write 4 Done Sent Done false
4253 2065 Acked Write 2 Done Sent Done false
4254 2066 Acked Write 3 Done Sent Done false
4255 2067 Acked Read 5 Done Sent Done false
4256 2068 Acked Write 6 Done Sent Done false
4257 2069 Acked Flush 0 Done Sent Done false
4258 2070 Acked Read 10 Done Sent Done false
4259 2071 Acked Read 3 Done Sent Done false
4260 2072 Acked Read 9 Done Sent Done false
4261 2073 Acked Read 4 Done Sent Done false
4262 2074 Acked Read 1 Done Sent Done false
4263 2075 Acked Write 4 Done Sent Done false
4264 2076 Acked Read 10 Done Sent Done false
4265 2077 Acked Read 5 Done Sent Done false
4266 2078 Acked Flush 0 Done Sent Done false
4267 2079 Acked Write 5 Done Sent Done false
4268 2080 Acked Write 8 Done Sent Done false
4269 2081 Acked Read 7 Done Sent Done false
4270 2082 Acked Write 9 Done Sent Done false
4271 2083 Acked Read 8 Done Sent Done false
4272 2084 Acked Read 10 Done Sent Done false
4273 2085 Acked Read 5 Done Sent Done false
4274 2086 Acked Read 3 Done Sent Done false
4275 2087 Acked Read 10 Done Sent Done false
4276 2088 Acked Flush 0 Done Sent Done false
4277 2089 Acked Read 4 Done Sent Done false
4278 2090 Acked Read 8 Done Sent Done false
4279 2091 Acked Read 9 Done Sent Done false
4280 2092 Acked Write 3 Done Sent Done false
4281 2093 Acked Read 6 Done Sent Done false
4282 2094 Acked Write 6 Done Sent Done false
4283 2095 Acked Write 6 Done Sent Done false
4284 2096 Acked Write 8 Done Sent Done false
4285 2097 Acked Write 9 Done Sent Done false
4286 2098 Acked Write 6 Done Sent Done false
4287 2099 Acked Write 8 Done Sent Done false
4288 2100 Acked Flush 0 Done Sent Done false
4289 2101 Acked Write 9 Done Sent Done false
4290 2102 Acked Write 4 Done Sent Done false
4291 2103 Acked Read 3 Done Sent Done false
4292 2104 Acked Read 6 Done Sent Done false
4293 2105 Acked Write 3 Done Sent Done false
4294 2106 Acked Read 4 Done Sent Done false
4295 2107 Acked Read 5 Done Sent Done false
4296 2108 Acked Read 7 Done Sent Done false
4297 2109 Acked Write 10 Done Sent Done false
4298 2110 Acked Write 10 Done Sent Done false
4299 2111 Acked Read 7 Done Sent Done false
4300 2112 Acked Read 2 Done Sent Done false
4301 2113 Acked Write 10 Done Sent Done false
4302 2114 Acked Flush 0 Done Sent Done false
4303 2115 Acked Write 8 Done Sent Done false
4304 2116 Acked Read 6 Done Sent Done false
4305 2117 Acked Read 5 Done Sent Done false
4306 2118 Acked Read 2 Done Sent Done false
4307 2119 Acked Write 1 Done Sent Done false
4308 2120 Acked Read 10 Done Sent Done false
4309 2121 Acked Write 8 Done Sent Done false
4310 2122 Acked Read 6 Done Sent Done false
4311 2123 Acked Read 8 Done Sent Done false
4312 2124 Acked Write 5 Done Sent Done false
4313 2125 Acked Read 3 Done Sent Done false
4314 2126 Acked Write 5 Done Sent Done false
4315 2127 Acked Write 8 Done Sent Done false
4316 2128 Acked Write 4 Done Sent Done false
4317 2129 Acked Write 3 Done Sent Done false
4318 2130 Acked Read 5 Done Sent Done false
4319 2131 Acked Read 3 Done Sent Done false
4320 2132 Acked Write 8 Done Sent Done false
4321 2133 Acked Write 5 Done Sent Done false
4322 2134 Acked Read 4 Done Sent Done false
4323 2135 Acked Read 4 Done Sent Done false
4324 2136 Acked Write 7 Done Sent Done false
4325 2137 Acked Write 6 Done Sent Done false
4326 2138 Acked Read 5 Done Sent Done false
4327 2139 Acked Read 9 Done Sent Done false
4328 2140 Acked Read 4 Done Sent Done false
4329 2141 Acked Write 8 Done Sent Done false
4330 2142 Acked Read 8 Done Sent Done false
4331 2143 Acked Write 10 Done Sent Done false
4332 2144 Acked Write 9 Done Sent Done false
4333 2145 Acked Read 8 Done Sent Done false
4334 2146 Acked Write 8 Done Sent Done false
4335 2147 Acked Read 6 Done Sent Done false
4336 2148 Acked Write 4 Done Sent Done false
4337 2149 Acked Write 3 Done Sent Done false
4338 2150 Acked Write 3 Done Sent Done false
4339 2151 Acked Read 7 Done Sent Done false
4340 2152 Acked Read 2 Done Sent Done false
4341 2153 Acked Read 1 Done Sent Done false
4342 2154 Acked Flush 0 Done Sent Done false
4343 2155 Acked Read 7 Done Sent Done false
4344 2156 Acked Read 6 Done Sent Done false
4345 2157 Acked Read 10 Done Sent Done false
4346 2158 Acked Read 9 Done Sent Done false
4347 2159 Acked Read 2 Done Sent Done false
4348 2160 Acked Read 1 Done Sent Done false
4349 2161 Acked Read 5 Done Sent Done false
4350 2162 Acked Read 5 Done Sent Done false
4351 2163 Acked Read 1 Done Sent Done false
4352 2164 Acked Write 4 Done Sent Done false
4353 2165 Acked Read 5 Done Sent Done false
4354 2166 Acked Read 9 Done Sent Done false
4355 2167 Acked Read 2 Done Sent Done false
4356 2168 Acked Read 2 Done Sent Done false
4357 2169 Acked Read 7 Done Sent Done false
4358 2170 Acked Read 10 Done Sent Done false
4359 2171 Acked Read 10 Done Sent Done false
4360 2172 Acked Read 1 Done Sent Done false
4361 2173 Acked Write 7 Done Sent Done false
4362 2174 Acked Write 7 Done Sent Done false
4363 2175 Acked Flush 0 Done Sent Done false
4364 2176 Acked Read 6 Done Sent Done false
4365 2177 Acked Write 8 Done Sent Done false
4366 2178 Acked Write 8 Done Sent Done false
4367 2179 Acked Read 5 Done Sent Done false
4368 2180 Acked Read 2 Done Sent Done false
4369 2181 Acked Write 6 Done Sent Done false
4370 2182 Acked Write 4 Done Sent Done false
4371 2183 Acked Write 1 Done Sent Done false
4372 2184 Acked Read 4 Done Sent Done false
4373 2185 Acked Read 3 Done Sent Done false
4374 2186 Acked Flush 0 Done Sent Done false
4375 2187 Acked Read 10 Done Sent Done false
4376 2188 Acked Write 8 Done Sent Done false
4377 2189 Acked Write 5 Done Sent Done false
4378 2190 Acked Read 6 Done Sent Done false
4379 2191 Acked Write 7 Done Sent Done false
4380 2192 Acked Write 4 Done Sent Done false
4381 2193 Acked Read 10 Done Sent Done false
4382 2194 Acked Write 1 Done Sent Done false
4383 2195 Acked Read 8 Done Sent Done false
4384 2196 Acked Read 2 Done Sent Done false
4385 2197 Acked Read 10 Done Sent Done false
4386 2198 Acked Read 1 Done Sent Done false
4387 2199 Acked Write 5 Done Sent Done false
4388 2200 Acked Write 7 Done Sent Done false
4389 2201 Acked Write 7 Done Sent Done false
4390 2202 Acked Read 5 Done Sent Done false
4391 2203 Acked Flush 0 Done Sent Done false
4392 2204 Acked Write 10 Done Sent Done false
4393 2205 Acked Read 5 Done Sent Done false
4394 2206 Acked Read 8 Done Sent Done false
4395 2207 Acked Read 8 Done Sent Done false
4396 2208 Acked Read 6 Done Sent Done false
4397 STATES DS:0 DS:1 DS:2 TOTAL
4398 Sent 0 302 0 302
4399 Done 302 0 302 604
4400 Skipped 0 0 0 0
4401 Error 0 0 0 0
4402 Last Flush: 2203 1906 2203
4403 Downstairs last five completed: 1906 1905 1904 1903 1902
4404 Upstairs last five completed: 2208 2207 2206 2205 2204
4405 CLIENT: Up:0 ds:302 act:2
4406 [8820] got start action
4407 [8820] I am going to restart
4408 Starting downstairs at port 8820
4409 Make output file at "/tmp/dsc/downstairs-8820.txt"
4410 [8820][1] reports Starting
4411 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1203)
4412 [8820][1] reports Running
4413 ----------------------------------------------------------------
4414 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
4415 States: Active Offline Active
4416 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4417 1907 Acked Write 9 Done Sent Done false
4418 1908 Acked Read 6 Done Sent Done false
4419 1909 Acked Read 7 Done Sent Done false
4420 1910 Acked Read 10 Done Sent Done false
4421 1911 Acked Flush 0 Done Sent Done false
4422 1912 Acked Write 8 Done Sent Done false
4423 1913 Acked Read 9 Done Sent Done false
4424 1914 Acked Write 8 Done Sent Done false
4425 1915 Acked Write 2 Done Sent Done false
4426 1916 Acked Read 2 Done Sent Done false
4427 1917 Acked Write 3 Done Sent Done false
4428 1918 Acked Write 6 Done Sent Done false
4429 1919 Acked Write 3 Done Sent Done false
4430 1920 Acked Read 3 Done Sent Done false
4431 1921 Acked Read 7 Done Sent Done false
4432 1922 Acked Read 8 Done Sent Done false
4433 1923 Acked Read 9 Done Sent Done false
4434 1924 Acked Read 7 Done Sent Done false
4435 1925 Acked Read 6 Done Sent Done false
4436 1926 Acked Read 5 Done Sent Done false
4437 1927 Acked Read 1 Done Sent Done false
4438 1928 Acked Write 7 Done Sent Done false
4439 1929 Acked Write 9 Done Sent Done false
4440 1930 Acked Write 3 Done Sent Done false
4441 1931 Acked Write 9 Done Sent Done false
4442 1932 Acked Read 10 Done Sent Done false
4443 1933 Acked Read 10 Done Sent Done false
4444 1934 Acked Read 2 Done Sent Done false
4445 1935 Acked Read 7 Done Sent Done false
4446 1936 Acked Write 4 Done Sent Done false
4447 1937 Acked Read 8 Done Sent Done false
4448 1938 Acked Read 3 Done Sent Done false
4449 1939 Acked Write 3 Done Sent Done false
4450 1940 Acked Flush 0 Done Sent Done false
4451 1941 Acked Read 1 Done Sent Done false
4452 1942 Acked Write 4 Done Sent Done false
4453 1943 Acked Write 2 Done Sent Done false
4454 1944 Acked Flush 0 Done Sent Done false
4455 1945 Acked Read 4 Done Sent Done false
4456 1946 Acked Read 7 Done Sent Done false
4457 1947 Acked Write 8 Done Sent Done false
4458 1948 Acked Read 8 Done Sent Done false
4459 1949 Acked Write 5 Done Sent Done false
4460 1950 Acked Write 5 Done Sent Done false
4461 1951 Acked Read 4 Done Sent Done false
4462 1952 Acked Write 6 Done Sent Done false
4463 1953 Acked Read 3 Done Sent Done false
4464 1954 Acked Write 9 Done Sent Done false
4465 1955 Acked Read 3 Done Sent Done false
4466 1956 Acked Read 1 Done Sent Done false
4467 1957 Acked Write 6 Done Sent Done false
4468 1958 Acked Write 8 Done Sent Done false
4469 1959 Acked Read 4 Done Sent Done false
4470 1960 Acked Read 8 Done Sent Done false
4471 1961 Acked Flush 0 Done Sent Done false
4472 1962 Acked Read 1 Done Sent Done false
4473 1963 Acked Read 2 Done Sent Done false
4474 1964 Acked Read 8 Done Sent Done false
4475 1965 Acked Read 9 Done Sent Done false
4476 1966 Acked Read 10 Done Sent Done false
4477 1967 Acked Read 10 Done Sent Done false
4478 1968 Acked Read 1 Done Sent Done false
4479 1969 Acked Flush 0 Done Sent Done false
4480 1970 Acked Read 1 Done Sent Done false
4481 1971 Acked Write 3 Done Sent Done false
4482 1972 Acked Read 5 Done Sent Done false
4483 1973 Acked Read 9 Done Sent Done false
4484 1974 Acked Write 10 Done Sent Done false
4485 1975 Acked Read 7 Done Sent Done false
4486 1976 Acked Read 9 Done Sent Done false
4487 1977 Acked Write 2 Done Sent Done false
4488 1978 Acked Read 6 Done Sent Done false
4489 1979 Acked Read 6 Done Sent Done false
4490 1980 Acked Read 10 Done Sent Done false
4491 1981 Acked Read 5 Done Sent Done false
4492 1982 Acked Write 2 Done Sent Done false
4493 1983 Acked Write 7 Done Sent Done false
4494 1984 Acked Write 8 Done Sent Done false
4495 1985 Acked Write 7 Done Sent Done false
4496 1986 Acked Write 6 Done Sent Done false
4497 1987 Acked Write 9 Done Sent Done false
4498 1988 Acked Read 8 Done Sent Done false
4499 1989 Acked Read 8 Done Sent Done false
4500 1990 Acked Read 5 Done Sent Done false
4501 1991 Acked Write 9 Done Sent Done false
4502 1992 Acked Flush 0 Done Sent Done false
4503 1993 Acked Read 6 Done Sent Done false
4504 1994 Acked Read 6 Done Sent Done false
4505 1995 Acked Read 9 Done Sent Done false
4506 1996 Acked Read 7 Done Sent Done false
4507 1997 Acked Read 1 Done Sent Done false
4508 1998 Acked Read 1 Done Sent Done false
4509 1999 Acked Flush 0 Done Sent Done false
4510 2000 Acked Write 4 Done Sent Done false
4511 2001 Acked Read 8 Done Sent Done false
4512 2002 Acked Read 9 Done Sent Done false
4513 2003 Acked Write 8 Done Sent Done false
4514 2004 Acked Read 10 Done Sent Done false
4515 2005 Acked Read 10 Done Sent Done false
4516 2006 Acked Read 3 Done Sent Done false
4517 2007 Acked Write 8 Done Sent Done false
4518 2008 Acked Write 3 Done Sent Done false
4519 2009 Acked Write 6 Done Sent Done false
4520 2010 Acked Write 10 Done Sent Done false
4521 2011 Acked Write 9 Done Sent Done false
4522 2012 Acked Write 8 Done Sent Done false
4523 2013 Acked Read 8 Done Sent Done false
4524 2014 Acked Read 5 Done Sent Done false
4525 2015 Acked Write 3 Done Sent Done false
4526 2016 Acked Read 4 Done Sent Done false
4527 2017 Acked Read 10 Done Sent Done false
4528 2018 Acked Write 9 Done Sent Done false
4529 2019 Acked Read 6 Done Sent Done false
4530 2020 Acked Write 4 Done Sent Done false
4531 2021 Acked Read 5 Done Sent Done false
4532 2022 Acked Write 4 Done Sent Done false
4533 2023 Acked Write 2 Done Sent Done false
4534 2024 Acked Flush 0 Done Sent Done false
4535 2025 Acked Read 10 Done Sent Done false
4536 2026 Acked Write 6 Done Sent Done false
4537 2027 Acked Read 4 Done Sent Done false
4538 2028 Acked Read 4 Done Sent Done false
4539 2029 Acked Read 5 Done Sent Done false
4540 2030 Acked Write 9 Done Sent Done false
4541 2031 Acked Flush 0 Done Sent Done false
4542 2032 Acked Flush 0 Done Sent Done false
4543 2033 Acked Read 1 Done Sent Done false
4544 2034 Acked Flush 0 Done Sent Done false
4545 2035 Acked Write 6 Done Sent Done false
4546 2036 Acked Write 10 Done Sent Done false
4547 2037 Acked Write 6 Done Sent Done false
4548 2038 Acked Write 8 Done Sent Done false
4549 2039 Acked Read 10 Done Sent Done false
4550 2040 Acked Read 8 Done Sent Done false
4551 2041 Acked Read 7 Done Sent Done false
4552 2042 Acked Write 5 Done Sent Done false
4553 2043 Acked Write 7 Done Sent Done false
4554 2044 Acked Read 3 Done Sent Done false
4555 2045 Acked Write 8 Done Sent Done false
4556 2046 Acked Write 1 Done Sent Done false
4557 2047 Acked Read 3 Done Sent Done false
4558 2048 Acked Read 2 Done Sent Done false
4559 2049 Acked Flush 0 Done Sent Done false
4560 2050 Acked Write 7 Done Sent Done false
4561 2051 Acked Write 1 Done Sent Done false
4562 2052 Acked Write 9 Done Sent Done false
4563 2053 Acked Read 8 Done Sent Done false
4564 2054 Acked Read 8 Done Sent Done false
4565 2055 Acked Read 2 Done Sent Done false
4566 2056 Acked Write 6 Done Sent Done false
4567 2057 Acked Write 9 Done Sent Done false
4568 2058 Acked Read 9 Done Sent Done false
4569 2059 Acked Write 8 Done Sent Done false
4570 2060 Acked Write 3 Done Sent Done false
4571 2061 Acked Write 9 Done Sent Done false
4572 2062 Acked Flush 0 Done Sent Done false
4573 2063 Acked Write 9 Done Sent Done false
4574 2064 Acked Write 4 Done Sent Done false
4575 2065 Acked Write 2 Done Sent Done false
4576 2066 Acked Write 3 Done Sent Done false
4577 2067 Acked Read 5 Done Sent Done false
4578 2068 Acked Write 6 Done Sent Done false
4579 2069 Acked Flush 0 Done Sent Done false
4580 2070 Acked Read 10 Done Sent Done false
4581 2071 Acked Read 3 Done Sent Done false
4582 2072 Acked Read 9 Done Sent Done false
4583 2073 Acked Read 4 Done Sent Done false
4584 2074 Acked Read 1 Done Sent Done false
4585 2075 Acked Write 4 Done Sent Done false
4586 2076 Acked Read 10 Done Sent Done false
4587 2077 Acked Read 5 Done Sent Done false
4588 2078 Acked Flush 0 Done Sent Done false
4589 2079 Acked Write 5 Done Sent Done false
4590 2080 Acked Write 8 Done Sent Done false
4591 2081 Acked Read 7 Done Sent Done false
4592 2082 Acked Write 9 Done Sent Done false
4593 2083 Acked Read 8 Done Sent Done false
4594 2084 Acked Read 10 Done Sent Done false
4595 2085 Acked Read 5 Done Sent Done false
4596 2086 Acked Read 3 Done Sent Done false
4597 2087 Acked Read 10 Done Sent Done false
4598 2088 Acked Flush 0 Done Sent Done false
4599 2089 Acked Read 4 Done Sent Done false
4600 2090 Acked Read 8 Done Sent Done false
4601 2091 Acked Read 9 Done Sent Done false
4602 2092 Acked Write 3 Done Sent Done false
4603 2093 Acked Read 6 Done Sent Done false
4604 2094 Acked Write 6 Done Sent Done false
4605 2095 Acked Write 6 Done Sent Done false
4606 2096 Acked Write 8 Done Sent Done false
4607 2097 Acked Write 9 Done Sent Done false
4608 2098 Acked Write 6 Done Sent Done false
4609 2099 Acked Write 8 Done Sent Done false
4610 2100 Acked Flush 0 Done Sent Done false
4611 2101 Acked Write 9 Done Sent Done false
4612 2102 Acked Write 4 Done Sent Done false
4613 2103 Acked Read 3 Done Sent Done false
4614 2104 Acked Read 6 Done Sent Done false
4615 2105 Acked Write 3 Done Sent Done false
4616 2106 Acked Read 4 Done Sent Done false
4617 2107 Acked Read 5 Done Sent Done false
4618 2108 Acked Read 7 Done Sent Done false
4619 2109 Acked Write 10 Done Sent Done false
4620 2110 Acked Write 10 Done Sent Done false
4621 2111 Acked Read 7 Done Sent Done false
4622 2112 Acked Read 2 Done Sent Done false
4623 2113 Acked Write 10 Done Sent Done false
4624 2114 Acked Flush 0 Done Sent Done false
4625 2115 Acked Write 8 Done Sent Done false
4626 2116 Acked Read 6 Done Sent Done false
4627 2117 Acked Read 5 Done Sent Done false
4628 2118 Acked Read 2 Done Sent Done false
4629 2119 Acked Write 1 Done Sent Done false
4630 2120 Acked Read 10 Done Sent Done false
4631 2121 Acked Write 8 Done Sent Done false
4632 2122 Acked Read 6 Done Sent Done false
4633 2123 Acked Read 8 Done Sent Done false
4634 2124 Acked Write 5 Done Sent Done false
4635 2125 Acked Read 3 Done Sent Done false
4636 2126 Acked Write 5 Done Sent Done false
4637 2127 Acked Write 8 Done Sent Done false
4638 2128 Acked Write 4 Done Sent Done false
4639 2129 Acked Write 3 Done Sent Done false
4640 2130 Acked Read 5 Done Sent Done false
4641 2131 Acked Read 3 Done Sent Done false
4642 2132 Acked Write 8 Done Sent Done false
4643 2133 Acked Write 5 Done Sent Done false
4644 2134 Acked Read 4 Done Sent Done false
4645 2135 Acked Read 4 Done Sent Done false
4646 2136 Acked Write 7 Done Sent Done false
4647 2137 Acked Write 6 Done Sent Done false
4648 2138 Acked Read 5 Done Sent Done false
4649 2139 Acked Read 9 Done Sent Done false
4650 2140 Acked Read 4 Done Sent Done false
4651 2141 Acked Write 8 Done Sent Done false
4652 2142 Acked Read 8 Done Sent Done false
4653 2143 Acked Write 10 Done Sent Done false
4654 2144 Acked Write 9 Done Sent Done false
4655 2145 Acked Read 8 Done Sent Done false
4656 2146 Acked Write 8 Done Sent Done false
4657 2147 Acked Read 6 Done Sent Done false
4658 2148 Acked Write 4 Done Sent Done false
4659 2149 Acked Write 3 Done Sent Done false
4660 2150 Acked Write 3 Done Sent Done false
4661 2151 Acked Read 7 Done Sent Done false
4662 2152 Acked Read 2 Done Sent Done false
4663 2153 Acked Read 1 Done Sent Done false
4664 2154 Acked Flush 0 Done Sent Done false
4665 2155 Acked Read 7 Done Sent Done false
4666 2156 Acked Read 6 Done Sent Done false
4667 2157 Acked Read 10 Done Sent Done false
4668 2158 Acked Read 9 Done Sent Done false
4669 2159 Acked Read 2 Done Sent Done false
4670 2160 Acked Read 1 Done Sent Done false
4671 2161 Acked Read 5 Done Sent Done false
4672 2162 Acked Read 5 Done Sent Done false
4673 2163 Acked Read 1 Done Sent Done false
4674 2164 Acked Write 4 Done Sent Done false
4675 2165 Acked Read 5 Done Sent Done false
4676 2166 Acked Read 9 Done Sent Done false
4677 2167 Acked Read 2 Done Sent Done false
4678 2168 Acked Read 2 Done Sent Done false
4679 2169 Acked Read 7 Done Sent Done false
4680 2170 Acked Read 10 Done Sent Done false
4681 2171 Acked Read 10 Done Sent Done false
4682 2172 Acked Read 1 Done Sent Done false
4683 2173 Acked Write 7 Done Sent Done false
4684 2174 Acked Write 7 Done Sent Done false
4685 2175 Acked Flush 0 Done Sent Done false
4686 2176 Acked Read 6 Done Sent Done false
4687 2177 Acked Write 8 Done Sent Done false
4688 2178 Acked Write 8 Done Sent Done false
4689 2179 Acked Read 5 Done Sent Done false
4690 2180 Acked Read 2 Done Sent Done false
4691 2181 Acked Write 6 Done Sent Done false
4692 2182 Acked Write 4 Done Sent Done false
4693 2183 Acked Write 1 Done Sent Done false
4694 2184 Acked Read 4 Done Sent Done false
4695 2185 Acked Read 3 Done Sent Done false
4696 2186 Acked Flush 0 Done Sent Done false
4697 2187 Acked Read 10 Done Sent Done false
4698 2188 Acked Write 8 Done Sent Done false
4699 2189 Acked Write 5 Done Sent Done false
4700 2190 Acked Read 6 Done Sent Done false
4701 2191 Acked Write 7 Done Sent Done false
4702 2192 Acked Write 4 Done Sent Done false
4703 2193 Acked Read 10 Done Sent Done false
4704 2194 Acked Write 1 Done Sent Done false
4705 2195 Acked Read 8 Done Sent Done false
4706 2196 Acked Read 2 Done Sent Done false
4707 2197 Acked Read 10 Done Sent Done false
4708 2198 Acked Read 1 Done Sent Done false
4709 2199 Acked Write 5 Done Sent Done false
4710 2200 Acked Write 7 Done Sent Done false
4711 2201 Acked Write 7 Done Sent Done false
4712 2202 Acked Read 5 Done Sent Done false
4713 2203 Acked Flush 0 Done Sent Done false
4714 2204 Acked Write 10 Done Sent Done false
4715 2205 Acked Read 5 Done Sent Done false
4716 2206 Acked Read 8 Done Sent Done false
4717 2207 Acked Read 8 Done Sent Done false
4718 2208 Acked Read 6 Done Sent Done false
4719 2209 Acked Flush 0 Done Sent Done false
4720 STATES DS:0 DS:1 DS:2 TOTAL
4721 Sent 0 303 0 303
4722 Done 303 0 303 606
4723 Skipped 0 0 0 0
4724 Error 0 0 0 0
4725 Last Flush: 2209 1906 2209
4726 Downstairs last five completed: 1906 1905 1904 1903 1902
4727 Upstairs last five completed: 2209 2208 2207 2206 2205
4728 CLIENT: Up:0 ds:303 act:2
47292024-11-07T14:04:16.684ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47302024-11-07T14:04:16.684ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:32980) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47312024-11-07T14:04:16.685ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47322024-11-07T14:04:16.685ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47332024-11-07T14:04:16.685ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47342024-11-07T14:04:16.685ZINFOcrucible: send last flush ID to this DS: 1906 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47352024-11-07T14:04:16.686ZINFOcrucible: Replied this last flush ID: 1906 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47362024-11-07T14:04:16.686ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47372024-11-07T14:04:16.686ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47382024-11-07T14:04:16.686ZINFOcrucible: [1] client re-new 303 jobs since flush 1906 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47392024-11-07T14:04:16.686ZINFOcrucible: [1] Replayed 303 jobs since flush: 1906 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
4740 ----------------------------------------------------------------
4741 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4742 Downstairs last five completed: 2209 2208 2207 2206 2205
4743 Upstairs last five completed: 2209 2208 2207 2206 2205
4744 CLIENT: Up:0 ds:0 act:3
4745 Replay: All jobs finished
4746 Main task has work to do, go find it
4747 got dsc Stop(0)
4748 stop 0
4749 Nov 07 14:04:18.976 INFO request completed[8810] Got stop action so:false kr:false
4750 , latency_us: 357, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 0ca8d587-5093-42d6-a5ba-d402cc443d1e, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
47512024-11-07T14:04:18.977ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47522024-11-07T14:04:18.977ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47532024-11-07T14:04:18.977ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47542024-11-07T14:04:18.977ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47552024-11-07T14:04:18.977ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
47562024-11-07T14:04:18.977ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
4757 Nov 07 14:04:18.977 INFO request completed, latency_us: 142, response_code: 200, uri: /state/cid/0, method: GET, req_id: 265953f4-df9b-4cf3-b6af-36ace5c03f07, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
4758 [8810] Exited with: ExitStatus(unix_wait_status(9))
4759 [8810][0] reports Exit
4760 Nov 07 14:04:22.978 INFO request completed, latency_us: 227, response_code: 200, uri: /state/cid/0, method: GET, req_id: f120278a-bfae-4494-9fd7-3f6942359730, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
4761 001/300 Read block 3130 len 20480
4762 002/300 Read block 86 len 40960
4763 003/300 Read block 4406 len 4096
4764 004/300 Write block 1643 len 24576 data: 2 3 3 3 3 3
4765 005/300 Read block 3366 len 40960
4766 006/300 Read block 4366 len 28672
4767 007/300 Read block 863 len 28672
4768 008/300 Read block 4837 len 20480
4769 009/300 Write block 3626 len 28672 data: 3 4 4 5 5 4 4
4770 010/300 Write block 1008 len 12288 data: 2 2 2
4771 011/300 Write block 1600 len 40960 data: 4 4 4 4 4 3 2 2 2 2
4772 012/300 Read block 2166 len 20480
4773 013/300 Write block 4989 len 4096 data: 3
4774 014/300 Write block 1159 len 20480 data: 2 2 2 2 2
4775 015/300 Write block 3703 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4776 016/300 Read block 3401 len 24576
4777 017/300 Read block 4596 len 20480
4778 018/300 Write block 4570 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4779 019/300 Read block 3267 len 20480
4780 020/300 Write block 1249 len 4096 data: 2
4781 021/300 Write block 2724 len 4096 data: 3
4782 022/300 Write block 4490 len 28672 data: 3 3 3 2 2 3 3
4783 023/300 Write block 1399 len 32768 data: 2 2 3 3 3 3 3 3
4784 024/300 Read block 3550 len 36864
4785 025/300 Flush
4786 026/300 Write block 2371 len 4096 data: 2
4787 027/300 Write block 4100 len 16384 data: 2 2 2 2
4788 028/300 Write block 4799 len 20480 data: 2 2 2 2 2
4789 029/300 Read block 446 len 32768
4790 030/300 Read block 3120 len 24576
4791 031/300 Write block 2862 len 28672 data: 3 3 2 2 2 2 2
4792 032/300 Read block 1107 len 12288
4793 033/300 Write block 2007 len 36864 data: 3 3 3 3 3 4 5 5 4
4794 034/300 Write block 2409 len 8192 data: 3 4
4795 035/300 Write block 3753 len 4096 data: 4
4796 036/300 Write block 3873 len 4096 data: 3
4797 037/300 Write block 3903 len 28672 data: 4 3 3 3 3 3 3
4798 038/300 Write block 422 len 32768 data: 3 3 2 2 2 2 4 4
4799 039/300 Read block 4570 len 4096
4800 040/300 Read block 3415 len 40960
4801 041/300 Write block 1354 len 24576 data: 2 2 3 3 3 4
4802 042/300 Write block 2321 len 16384 data: 2 2 2 2
4803 043/300 Write block 3387 len 8192 data: 2 2
4804 044/300 Write block 2620 len 4096 data: 3
4805 045/300 Read block 4234 len 16384
4806 046/300 Flush
4807 047/300 Write block 1297 len 28672 data: 2 2 2 2 2 2 2
4808 048/300 Write block 2544 len 36864 data: 5 4 3 3 3 2 2 2 2
4809 049/300 Write block 4283 len 24576 data: 2 2 2 2 2 3
4810 050/300 Write block 1264 len 16384 data: 2 2 2 2
4811 051/300 Read block 437 len 40960
4812 052/300 Write block 1066 len 36864 data: 2 2 2 2 2 2 2 2 2
4813 053/300 Read block 1758 len 20480
4814 054/300 Read block 2654 len 16384
4815 055/300 Read block 4621 len 40960
4816 056/300 Read block 3736 len 4096
4817 057/300 Flush
4818 058/300 Read block 3053 len 8192
4819 059/300 Flush
4820 060/300 Write block 4629 len 32768 data: 3 3 3 3 3 3 3 2
4821 061/300 Write block 4068 len 8192 data: 2 2
4822 062/300 Write block 4251 len 32768 data: 3 3 3 3 3 3 2 2
4823 063/300 Write block 2472 len 32768 data: 2 2 2 3 3 3 2 2
4824 064/300 Flush
4825 065/300 Read block 3534 len 24576
4826 066/300 Read block 3125 len 4096
4827 067/300 Read block 639 len 16384
4828 068/300 Read block 1588 len 40960
4829 069/300 Read block 703 len 24576
4830 070/300 Flush
4831 071/300 Flush
4832 072/300 Read block 2289 len 12288
4833 073/300 Write block 3011 len 12288 data: 2 2 2
4834 074/300 Read block 1334 len 20480
4835 075/300 Flush
4836 076/300 Read block 2470 len 40960
4837 077/300 Write block 1284 len 12288 data: 2 2 2
4838 078/300 Read block 1699 len 28672
4839 079/300 Read block 1914 len 36864
4840 080/300 Write block 4314 len 28672 data: 3 3 3 3 3 3 3
4841 081/300 Write block 2499 len 4096 data: 2
4842 082/300 Write block 2276 len 8192 data: 2 2
4843 083/300 Flush
4844 084/300 Read block 1326 len 24576
4845 085/300 Read block 184 len 12288
4846 086/300 Read block 1794 len 40960
4847 087/300 Flush
4848 088/300 Write block 1138 len 8192 data: 2 2
4849 089/300 Write block 4733 len 40960 data: 2 2 2 2 2 2 2 2 3 3
4850 090/300 Write block 951 len 24576 data: 3 3 3 3 3 3
4851 091/300 Read block 2069 len 36864
4852 092/300 Read block 1457 len 36864
4853 093/300 Read block 3301 len 32768
4854 094/300 Read block 2616 len 36864
4855 095/300 Read block 855 len 28672
4856 096/300 Read block 2972 len 24576
4857 097/300 Read block 2038 len 16384
4858 098/300 Read block 3747 len 36864
4859 099/300 Write block 515 len 28672 data: 2 2 2 3 3 3 3
4860 100/300 Read block 1454 len 24576
4861 101/300 Flush
4862 102/300 Read block 450 len 36864
4863 103/300 Write block 2829 len 4096 data: 4
4864 104/300 Write block 4636 len 16384 data: 3 2 2 2
4865 105/300 Read block 3613 len 24576
4866 106/300 Write block 3282 len 4096 data: 3
4867 107/300 Read block 1094 len 8192
4868 108/300 Write block 2205 len 20480 data: 2 2 2 2 2
4869 109/300 Read block 1867 len 16384
4870 110/300 Read block 4199 len 36864
4871 111/300 Write block 1984 len 16384 data: 2 2 2 2
4872 112/300 Write block 4733 len 36864 data: 3 3 3 3 3 3 3 3 4
4873 113/300 Read block 4759 len 12288
4874 114/300 Flush
4875 115/300 Flush
4876 116/300 Read block 3768 len 24576
4877 117/300 Read block 3930 len 40960
4878 118/300 Read block 3184 len 36864
4879 119/300 Read block 3147 len 32768
4880 120/300 Flush
4881 121/300 Write block 2853 len 36864 data: 2 3 3 3 3 3 3 3 3
4882 122/300 Flush
4883 123/300 Read block 923 len 24576
4884 124/300 Write block 3503 len 4096 data: 2
4885 125/300 Read block 3837 len 12288
4886 126/300 Read block 3030 len 28672
4887 127/300 Read block 4099 len 12288
4888 128/300 Write block 2622 len 12288 data: 2 2 2
4889 129/300 Write block 1442 len 28672 data: 6 6 5 4 4 4 4
4890 130/300 Write block 1675 len 4096 data: 3
4891 131/300 Read block 4015 len 12288
4892 132/300 Read block 507 len 36864
4893 133/300 Write block 1206 len 36864 data: 2 2 2 2 2 2 2 2 2
4894 134/300 Read block 2677 len 12288
4895 135/300 Read block 536 len 20480
4896 136/300 Write block 3472 len 12288 data: 2 2 2
4897 137/300 Read block 2098 len 36864
4898 138/300 Write block 400 len 24576 data: 2 2 3 3 3 3
4899 139/300 Read block 4296 len 36864
4900 140/300 Write block 2905 len 40960 data: 2 2 2 2 2 2 3 3 2 2
4901 141/300 Read block 80 len 24576
4902 142/300 Read block 3622 len 28672
4903 143/300 Write block 3717 len 28672 data: 2 2 2 3 3 3 2
4904 144/300 Read block 3946 len 16384
4905 145/300 Flush
4906 146/300 Write block 1152 len 40960 data: 2 2 2 2 2 2 2 3 3 3
4907 147/300 Read block 290 len 28672
4908 148/300 Write block 4435 len 28672 data: 3 3 3 3 3 2 2
4909 149/300 Read block 1202 len 36864
4910 150/300 Read block 4400 len 16384
4911 151/300 Write block 344 len 8192 data: 3 3
4912 152/300 Write block 4099 len 32768 data: 2 3 3 3 3 2 2 2
4913 153/300 Write block 1573 len 36864 data: 3 3 3 3 2 2 2 2 2
4914 154/300 Read block 424 len 28672
4915 155/300 Read block 3902 len 28672
4916 156/300 Write block 4727 len 28672 data: 2 2 2 2 2 2 4
4917 157/300 Read block 3048 len 28672
4918 158/300 Flush
4919 159/300 Read block 2637 len 20480
4920 160/300 Read block 1369 len 16384
4921 161/300 Read block 3878 len 8192
4922 162/300 Write block 2719 len 32768 data: 2 2 3 3 3 4 3 3
4923 163/300 Read block 2006 len 36864
4924 164/300 Read block 1567 len 32768
4925 165/300 Read block 4744 len 40960
4926 166/300 Write block 301 len 20480 data: 3 2 2 2 3
4927 167/300 Read block 3177 len 24576
4928 168/300 Read block 1883 len 12288
4929 169/300 Flush
4930 170/300 Write block 2300 len 36864 data: 2 2 2 2 3 3 3 3 3
4931 171/300 Write block 4247 len 8192 data: 2 2
4932 172/300 Flush
4933 173/300 Write block 3265 len 12288 data: 2 2 2
4934 174/300 Read block 3449 len 8192
4935 175/300 Write block 3743 len 12288 data: 3 2 2
4936 176/300 Write block 2437 len 20480 data: 3 3 3 3 3
4937 177/300 Write block 2622 len 4096 data: 3
4938 178/300 Read block 2049 len 24576
4939 179/300 Write block 3267 len 28672 data: 3 2 2 2 2 3 3
4940 180/300 Read block 3202 len 28672
4941 181/300 Flush
4942 182/300 Read block 1035 len 20480
4943 183/300 Write block 4070 len 8192 data: 2 2
4944 184/300 Read block 4747 len 8192
4945 185/300 Read block 1317 len 20480
4946 186/300 Read block 1159 len 36864
4947 187/300 Read block 1168 len 8192
4948 188/300 Read block 400 len 4096
4949 189/300 Write block 692 len 24576 data: 2 2 2 3 3 3
4950 190/300 Read block 2504 len 40960
4951 191/300 Write block 4918 len 40960 data: 3 3 3 2 2 2 2 2 2 2
4952 192/300 Write block 3838 len 28672 data: 2 2 2 2 3 3 3
4953 193/300 Write block 3939 len 8192 data: 2 2
4954 194/300 Read block 505 len 8192
4955 195/300 Write block 4041 len 32768 data: 2 2 2 2 2 2 2 2
4956 196/300 Write block 1768 len 16384 data: 2 2 2 2
4957 197/300 Read block 71 len 32768
4958 198/300 Read block 1407 len 20480
4959 199/300 Write block 4701 len 12288 data: 3 2 2
4960 200/300 Read block 3949 len 28672
4961 201/300 Read block 3970 len 32768
4962 202/300 Read block 3507 len 4096
4963 203/300 Read block 1162 len 32768
4964 204/300 Write block 3630 len 4096 data: 6
4965 205/300 Read block 823 len 16384
4966 206/300 Read block 2936 len 40960
4967 207/300 Read block 487 len 24576
4968 208/300 Write block 1875 len 20480 data: 2 2 2 2 2
4969 209/300 Read block 3655 len 8192
4970 210/300 Read block 1096 len 32768
4971 211/300 Read block 2358 len 24576
4972 212/300 Flush
4973 213/300 Read block 2941 len 40960
4974 214/300 Read block 3007 len 16384
4975 215/300 Read block 1317 len 24576
4976 216/300 Read block 4766 len 12288
4977 217/300 Read block 717 len 16384
4978 218/300 Write block 1509 len 16384 data: 3 2 2 2
4979 219/300 Read block 4075 len 40960
4980 220/300 Read block 2783 len 28672
4981 221/300 Write block 2277 len 16384 data: 3 2 2 2
4982 222/300 Read block 274 len 20480
4983 223/300 Write block 1560 len 20480 data: 3 3 2 2 2
4984 224/300 Read block 3327 len 32768
4985 225/300 Read block 2027 len 36864
4986 226/300 Read block 3812 len 4096
4987 227/300 Write block 3796 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4988 228/300 Write block 4950 len 4096 data: 2
4989 229/300 Write block 1675 len 8192 data: 4 3
4990 230/300 Read block 4805 len 40960
4991 231/300 Read block 3561 len 40960
4992 232/300 Read block 540 len 32768
4993 233/300 Read block 3566 len 32768
4994 234/300 Write block 140 len 16384 data: 2 2 2 2
4995 235/300 Write block 2548 len 8192 data: 4 3
4996 236/300 Write block 2571 len 40960 data: 3 2 2 2 2 2 2 2 2 2
4997 237/300 Read block 464 len 4096
4998 238/300 Read block 2601 len 24576
4999 239/300 Read block 710 len 8192
5000 240/300 Read block 828 len 16384
5001 241/300 Read block 264 len 4096
5002 242/300 Read block 1755 len 40960
5003 243/300 Write block 4021 len 12288 data: 3 3 3
5004 244/300 Flush
5005 245/300 Read block 88 len 28672
5006 246/300 Write block 3424 len 28672 data: 4 4 4 4 4 4 4
5007 247/300 Write block 2805 len 20480 data: 3 3 3 3 3
5008 248/300 Read block 1569 len 24576
5009 249/300 Write block 4519 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5010 250/300 Write block 1186 len 28672 data: 2 2 3 3 3 3 3
5011 251/300 Flush
5012 252/300 Write block 1039 len 28672 data: 2 2 2 2 2 2 2
5013 253/300 Read block 4445 len 24576
5014 254/300 Write block 790 len 28672 data: 3 2 2 2 2 2 2
5015 255/300 Write block 1742 len 40960 data: 2 2 2 2 2 2 2 2 3 3
5016 256/300 Read block 1288 len 28672
5017 257/300 Write block 4188 len 12288 data: 2 3 3
5018 258/300 Write block 3761 len 16384 data: 2 2 2 2
5019 259/300 Read block 3404 len 16384
5020 260/300 Write block 4987 len 8192 data: 2 2
5021 261/300 Read block 2351 len 20480
5022 262/300 Read block 3567 len 40960
5023 263/300 Read block 4614 len 16384
5024 264/300 Write block 4842 len 24576 data: 4 4 4 3 3 3
5025 265/300 Read block 4632 len 24576
5026 266/300 Write block 2554 len 16384 data: 2 2 2 2
5027 267/300 Flush
5028 268/300 Write block 1041 len 36864 data: 3 3 3 3 3 2 2 2 2
5029 269/300 Flush
5030 270/300 Read block 4762 len 32768
5031 271/300 Read block 2789 len 16384
5032 272/300 Read block 778 len 28672
5033 273/300 Write block 3900 len 20480 data: 4 4 3 5 4
5034 274/300 Read block 4154 len 12288
5035 275/300 Write block 648 len 36864 data: 3 2 2 2 2 2 2 2 2
5036 276/300 Write block 1017 len 8192 data: 3 3
5037 277/300 Read block 349 len 28672
5038 278/300 Read block 1478 len 36864
5039 279/300 Write block 2184 len 12288 data: 3 3 3
5040 280/300 Read block 1247 len 12288
5041 281/300 Read block 1326 len 24576
5042 282/300 Write block 3352 len 24576 data: 3 3 3 3 3 3
5043 283/300 Read block 4814 len 20480
5044 284/300 Write block 2563 len 20480 data: 3 3 4 4 5
5045 285/300 Write block 4602 len 16384 data: 4 4 4 3
5046 286/300 Write block 4105 len 36864 data: 3 3 2 2 2 2 2 2 2
5047 287/300 Read block 3875 len 16384
5048 288/300 Flush
5049 289/300 Write block 1354 len 28672 data: 3 3 4 4 4 5 4
5050 290/300 Read block 454 len 40960
5051 291/300 Read block 327 len 4096
5052 292/300 Read block 4511 len 16384
5053 293/300 Write block 3456 len 28672 data: 2 2 2 2 2 2 2
5054 294/300 Read block 3725 len 32768
5055 295/300 Read block 1822 len 40960
5056 296/300 Write block 2551 len 4096 data: 3
5057 297/300 Write block 1902 len 36864 data: 2 2 2 2 2 2 2 2 2
5058 298/300 Write block 1966 len 36864 data: 2 3 3 3 3 3 3 3 3
5059 299/300 Read block 1932 len 20480
5060 300/300 Write block 4809 len 40960 data: 2 2 2 2 2 2 2 2 2 3
5061 301/300 Read block 4460 len 40960
5062 Main task has work to do, go find it
5063 got dsc Start(0)
5064 start 0
5065 Nov 07 14:04:23.266 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/0, method: POST, req_id: efceb3f9-5460-4773-996f-583b27e006a2, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
5066 Replay: started 0, returned:Ok(())
5067 ----------------------------------------------------------------
5068 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5069 States: Offline Active Active
5070 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5071 2210 Acked Read 5 Sent Done Done false
5072 2211 Acked Read 10 Sent Done Done false
5073 2212 Acked Read 1 Sent Done Done false
5074 2213 Acked Write 6 Sent Done Done false
5075 2214 Acked Read 10 Sent Done Done false
5076 2215 Acked Read 7 Sent Done Done false
5077 2216 Acked Read 7 Sent Done Done false
5078 2217 Acked Read 5 Sent Done Done false
5079 2218 Acked Write 7 Sent Done Done false
5080 2219 Acked Write 3 Sent Done Done false
5081 2220 Acked Write 10 Sent Done Done false
5082 2221 Acked Read 5 Sent Done Done false
5083 2222 Acked Write 1 Sent Done Done false
5084 2223 Acked Write 5 Sent Done Done false
5085 2224 Acked Write 10 Sent Done Done false
5086 2225 Acked Read 6 Sent Done Done false
5087 2226 Acked Read 5 Sent Done Done false
5088 2227 Acked Write 10 Sent Done Done false
5089 2228 Acked Read 5 Sent Done Done false
5090 2229 Acked Write 1 Sent Done Done false
5091 2230 Acked Write 1 Sent Done Done false
5092 2231 Acked Write 7 Sent Done Done false
5093 2232 Acked Write 8 Sent Done Done false
5094 2233 Acked Read 9 Sent Done Done false
5095 2234 Acked Flush 0 Sent Done Done false
5096 2235 Acked Write 1 Sent Done Done false
5097 2236 Acked Write 4 Sent Done Done false
5098 2237 Acked Write 5 Sent Done Done false
5099 2238 Acked Read 8 Sent Done Done false
5100 2239 Acked Read 6 Sent Done Done false
5101 2240 Acked Write 7 Sent Done Done false
5102 2241 Acked Read 3 Sent Done Done false
5103 2242 Acked Write 9 Sent Done Done false
5104 2243 Acked Write 2 Sent Done Done false
5105 2244 Acked Write 1 Sent Done Done false
5106 2245 Acked Write 1 Sent Done Done false
5107 2246 Acked Write 7 Sent Done Done false
5108 2247 Acked Write 8 Sent Done Done false
5109 2248 Acked Read 1 Sent Done Done false
5110 2249 Acked Read 10 Sent Done Done false
5111 2250 Acked Write 6 Sent Done Done false
5112 2251 Acked Write 4 Sent Done Done false
5113 2252 Acked Write 2 Sent Done Done false
5114 2253 Acked Write 1 Sent Done Done false
5115 2254 Acked Read 4 Sent Done Done false
5116 2255 Acked Flush 0 Sent Done Done false
5117 2256 Acked Write 7 Sent Done Done false
5118 2257 Acked Write 9 Sent Done Done false
5119 2258 Acked Write 6 Sent Done Done false
5120 2259 Acked Write 4 Sent Done Done false
5121 2260 Acked Read 10 Sent Done Done false
5122 2261 Acked Write 9 Sent Done Done false
5123 2262 Acked Read 5 Sent Done Done false
5124 2263 Acked Read 4 Sent Done Done false
5125 2264 Acked Read 10 Sent Done Done false
5126 2265 Acked Read 1 Sent Done Done false
5127 2266 Acked Flush 0 Sent Done Done false
5128 2267 Acked Read 2 Sent Done Done false
5129 2268 Acked Flush 0 Sent Done Done false
5130 2269 Acked Write 8 Sent Done Done false
5131 2270 Acked Write 2 Sent Done Done false
5132 2271 Acked Write 8 Sent Done Done false
5133 2272 Acked Write 8 Sent Done Done false
5134 2273 Acked Flush 0 Sent Done Done false
5135 2274 Acked Read 6 Sent Done Done false
5136 2275 Acked Read 1 Sent Done Done false
5137 2276 Acked Read 4 Sent Done Done false
5138 2277 Acked Read 10 Sent Done Done false
5139 2278 Acked Read 6 Sent Done Done false
5140 2279 Acked Flush 0 Sent Done Done false
5141 2280 Acked Flush 0 Sent Done Done false
5142 2281 Acked Read 3 Sent Done Done false
5143 2282 Acked Write 3 Sent Done Done false
5144 2283 Acked Read 5 Sent Done Done false
5145 2284 Acked Flush 0 Sent Done Done false
5146 2285 Acked Read 10 Sent Done Done false
5147 2286 Acked Write 3 Sent Done Done false
5148 2287 Acked Read 7 Sent Done Done false
5149 2288 Acked Read 9 Sent Done Done false
5150 2289 Acked Write 7 Sent Done Done false
5151 2290 Acked Write 1 Sent Done Done false
5152 2291 Acked Write 2 Sent Done Done false
5153 2292 Acked Flush 0 Sent Done Done false
5154 2293 Acked Read 6 Sent Done Done false
5155 2294 Acked Read 3 Sent Done Done false
5156 2295 Acked Read 10 Sent Done Done false
5157 2296 Acked Flush 0 Sent Done Done false
5158 2297 Acked Write 2 Sent Done Done false
5159 2298 Acked Write 10 Sent Done Done false
5160 2299 Acked Write 6 Sent Done Done false
5161 2300 Acked Read 9 Sent Done Done false
5162 2301 Acked Read 9 Sent Done Done false
5163 2302 Acked Read 8 Sent Done Done false
5164 2303 Acked Read 9 Sent Done Done false
5165 2304 Acked Read 7 Sent Done Done false
5166 2305 Acked Read 6 Sent Done Done false
5167 2306 Acked Read 4 Sent Done Done false
5168 2307 Acked Read 9 Sent Done Done false
5169 2308 Acked Write 7 Sent Done Done false
5170 2309 Acked Read 6 Sent Done Done false
5171 2310 Acked Flush 0 Sent Done Done false
5172 2311 Acked Read 9 Sent Done Done false
5173 2312 Acked Write 1 Sent Done Done false
5174 2313 Acked Write 4 Sent Done Done false
5175 2314 Acked Read 6 Sent Done Done false
5176 2315 Acked Write 1 Sent Done Done false
5177 2316 Acked Read 2 Sent Done Done false
5178 2317 Acked Write 5 Sent Done Done false
5179 2318 Acked Read 4 Sent Done Done false
5180 2319 Acked Read 9 Sent Done Done false
5181 2320 Acked Write 4 Sent Done Done false
5182 2321 Acked Write 9 Sent Done Done false
5183 2322 Acked Read 3 Sent Done Done false
5184 2323 Acked Flush 0 Sent Done Done false
5185 2324 Acked Flush 0 Sent Done Done false
5186 2325 Acked Read 6 Sent Done Done false
5187 2326 Acked Read 10 Sent Done Done false
5188 2327 Acked Read 9 Sent Done Done false
5189 2328 Acked Read 8 Sent Done Done false
5190 2329 Acked Flush 0 Sent Done Done false
5191 2330 Acked Write 9 Sent Done Done false
5192 2331 Acked Flush 0 Sent Done Done false
5193 2332 Acked Read 6 Sent Done Done false
5194 2333 Acked Write 1 Sent Done Done false
5195 2334 Acked Read 3 Sent Done Done false
5196 2335 Acked Read 7 Sent Done Done false
5197 2336 Acked Read 3 Sent Done Done false
5198 2337 Acked Write 3 Sent Done Done false
5199 2338 Acked Write 7 Sent Done Done false
5200 2339 Acked Write 1 Sent Done Done false
5201 2340 Acked Read 3 Sent Done Done false
5202 2341 Acked Read 9 Sent Done Done false
5203 2342 Acked Write 9 Sent Done Done false
5204 2343 Acked Read 3 Sent Done Done false
5205 2344 Acked Read 5 Sent Done Done false
5206 2345 Acked Write 3 Sent Done Done false
5207 2346 Acked Read 9 Sent Done Done false
5208 2347 Acked Write 6 Sent Done Done false
5209 2348 Acked Read 9 Sent Done Done false
5210 2349 Acked Write 10 Sent Done Done false
5211 2350 Acked Read 6 Sent Done Done false
5212 2351 Acked Read 7 Sent Done Done false
5213 2352 Acked Write 7 Sent Done Done false
5214 2353 Acked Read 4 Sent Done Done false
5215 2354 Acked Flush 0 Sent Done Done false
5216 2355 Acked Write 10 Sent Done Done false
5217 2356 Acked Read 7 Sent Done Done false
5218 2357 Acked Write 7 Sent Done Done false
5219 2358 Acked Read 9 Sent Done Done false
5220 2359 Acked Read 4 Sent Done Done false
5221 2360 Acked Write 2 Sent Done Done false
5222 2361 Acked Write 8 Sent Done Done false
5223 2362 Acked Write 9 Sent Done Done false
5224 2363 Acked Read 7 Sent Done Done false
5225 2364 Acked Read 7 Sent Done Done false
5226 2365 Acked Write 7 Sent Done Done false
5227 2366 Acked Read 7 Sent Done Done false
5228 2367 Acked Flush 0 Sent Done Done false
5229 2368 Acked Read 5 Sent Done Done false
5230 2369 Acked Read 4 Sent Done Done false
5231 2370 Acked Read 2 Sent Done Done false
5232 2371 Acked Write 8 Sent Done Done false
5233 2372 Acked Read 9 Sent Done Done false
5234 2373 Acked Read 8 Sent Done Done false
5235 2374 Acked Read 10 Sent Done Done false
5236 2375 Acked Write 5 Sent Done Done false
5237 2376 Acked Read 6 Sent Done Done false
5238 2377 Acked Read 3 Sent Done Done false
5239 2378 Acked Flush 0 Sent Done Done false
5240 2379 Acked Write 9 Sent Done Done false
5241 2380 Acked Write 2 Sent Done Done false
5242 2381 Acked Flush 0 Sent Done Done false
5243 2382 Acked Write 3 Sent Done Done false
5244 2383 Acked Read 2 Sent Done Done false
5245 2384 Acked Write 3 Sent Done Done false
5246 2385 Acked Write 5 Sent Done Done false
5247 2386 Acked Write 1 Sent Done Done false
5248 2387 Acked Read 6 Sent Done Done false
5249 2388 Acked Write 7 Sent Done Done false
5250 2389 Acked Read 7 Sent Done Done false
5251 2390 Acked Flush 0 Sent Done Done false
5252 2391 Acked Read 5 Sent Done Done false
5253 2392 Acked Write 2 Sent Done Done false
5254 2393 Acked Read 2 Sent Done Done false
5255 2394 Acked Read 5 Sent Done Done false
5256 2395 Acked Read 9 Sent Done Done false
5257 2396 Acked Read 2 Sent Done Done false
5258 2397 Acked Read 1 Sent Done Done false
5259 2398 Acked Write 6 Sent Done Done false
5260 2399 Acked Read 10 Sent Done Done false
5261 2400 Acked Write 10 Sent Done Done false
5262 2401 Acked Write 7 Sent Done Done false
5263 2402 Acked Write 2 Sent Done Done false
5264 2403 Acked Read 2 Sent Done Done false
5265 2404 Acked Write 8 Sent Done Done false
5266 2405 Acked Write 4 Sent Done Done false
5267 2406 Acked Read 8 Sent Done Done false
5268 2407 Acked Read 5 Sent Done Done false
5269 2408 Acked Write 3 Sent Done Done false
5270 2409 Acked Read 7 Sent Done Done false
5271 2410 Acked Read 8 Sent Done Done false
5272 2411 Acked Read 1 Sent Done Done false
5273 2412 Acked Read 8 Sent Done Done false
5274 2413 Acked Write 1 Sent Done Done false
5275 2414 Acked Read 4 Sent Done Done false
5276 2415 Acked Read 10 Sent Done Done false
5277 2416 Acked Read 6 Sent Done Done false
5278 2417 Acked Write 5 Sent Done Done false
5279 2418 Acked Read 2 Sent Done Done false
5280 2419 Acked Read 8 Sent Done Done false
5281 2420 Acked Read 6 Sent Done Done false
5282 2421 Acked Flush 0 Sent Done Done false
5283 2422 Acked Read 10 Sent Done Done false
5284 2423 Acked Read 4 Sent Done Done false
5285 2424 Acked Read 6 Sent Done Done false
5286 2425 Acked Read 3 Sent Done Done false
5287 2426 Acked Read 4 Sent Done Done false
5288 2427 Acked Write 4 Sent Done Done false
5289 2428 Acked Read 10 Sent Done Done false
5290 2429 Acked Read 7 Sent Done Done false
5291 2430 Acked Write 4 Sent Done Done false
5292 2431 Acked Read 5 Sent Done Done false
5293 2432 Acked Write 5 Sent Done Done false
5294 2433 Acked Read 8 Sent Done Done false
5295 2434 Acked Read 9 Sent Done Done false
5296 2435 Acked Read 1 Sent Done Done false
5297 2436 Acked Write 10 Sent Done Done false
5298 2437 Acked Write 1 Sent Done Done false
5299 2438 Acked Write 2 Sent Done Done false
5300 2439 Acked Read 10 Sent Done Done false
5301 2440 Acked Read 10 Sent Done Done false
5302 2441 Acked Read 8 Sent Done Done false
5303 2442 Acked Read 8 Sent Done Done false
5304 2443 Acked Write 4 Sent Done Done false
5305 2444 Acked Write 2 Sent Done Done false
5306 2445 Acked Write 10 Sent Done Done false
5307 2446 Acked Read 1 Sent Done Done false
5308 2447 Acked Read 6 Sent Done Done false
5309 2448 Acked Read 2 Sent Done Done false
5310 2449 Acked Read 4 Sent Done Done false
5311 2450 Acked Read 1 Sent Done Done false
5312 2451 Acked Read 10 Sent Done Done false
5313 2452 Acked Write 3 Sent Done Done false
5314 2453 Acked Flush 0 Sent Done Done false
5315 2454 Acked Read 7 Sent Done Done false
5316 2455 Acked Write 7 Sent Done Done false
5317 2456 Acked Write 5 Sent Done Done false
5318 2457 Acked Read 6 Sent Done Done false
5319 2458 Acked Write 10 Sent Done Done false
5320 2459 Acked Write 7 Sent Done Done false
5321 2460 Acked Flush 0 Sent Done Done false
5322 2461 Acked Write 7 Sent Done Done false
5323 2462 Acked Read 6 Sent Done Done false
5324 2463 Acked Write 7 Sent Done Done false
5325 2464 Acked Write 10 Sent Done Done false
5326 2465 Acked Read 7 Sent Done Done false
5327 2466 Acked Write 3 Sent Done Done false
5328 2467 Acked Write 4 Sent Done Done false
5329 2468 Acked Read 4 Sent Done Done false
5330 2469 Acked Write 2 Sent Done Done false
5331 2470 Acked Read 5 Sent Done Done false
5332 2471 Acked Read 10 Sent Done Done false
5333 2472 Acked Read 4 Sent Done Done false
5334 2473 Acked Write 6 Sent Done Done false
5335 2474 Acked Read 6 Sent Done Done false
5336 2475 Acked Write 4 Sent Done Done false
5337 2476 Acked Flush 0 Sent Done Done false
5338 2477 Acked Write 9 Sent Done Done false
5339 2478 Acked Flush 0 Sent Done Done false
5340 2479 Acked Read 8 Sent Done Done false
5341 2480 Acked Read 4 Sent Done Done false
5342 2481 Acked Read 7 Sent Done Done false
5343 2482 Acked Write 5 Sent Done Done false
5344 2483 Acked Read 3 Sent Done Done false
5345 2484 Acked Write 9 Sent Done Done false
5346 2485 Acked Write 2 Sent Done Done false
5347 2486 Acked Read 7 Sent Done Done false
5348 2487 Acked Read 9 Sent Done Done false
5349 2488 Acked Write 3 Sent Done Done false
5350 2489 Acked Read 3 Sent Done Done false
5351 2490 Acked Read 6 Sent Done Done false
5352 2491 Acked Write 6 Sent Done Done false
5353 2492 Acked Read 5 Sent Done Done false
5354 2493 Acked Write 5 Sent Done Done false
5355 2494 Acked Write 4 Sent Done Done false
5356 2495 Acked Write 9 Sent Done Done false
5357 2496 Acked Read 4 Sent Done Done false
5358 2497 Acked Flush 0 Sent Done Done false
5359 2498 Acked Write 7 Sent Done Done false
5360 2499 Acked Read 10 Sent Done Done false
5361 2500 Acked Read 1 Sent Done Done false
5362 2501 Acked Read 4 Sent Done Done false
5363 2502 Acked Write 7 Sent Done Done false
5364 2503 Acked Read 8 Sent Done Done false
5365 2504 Acked Read 10 Sent Done Done false
5366 2505 Acked Write 1 Sent Done Done false
5367 2506 Acked Write 9 Sent Done Done false
5368 2507 Acked Write 9 Sent Done Done false
5369 2508 Acked Read 5 Sent Done Done false
5370 2509 Acked Write 10 Sent Done Done false
5371 2510 Acked Read 10 Sent Done Done false
5372 STATES DS:0 DS:1 DS:2 TOTAL
5373 Sent 301 0 0 301
5374 Done 0 301 301 602
5375 Skipped 0 0 0 0
5376 Error 0 0 0 0
5377 Last Flush: 2209 2497 2497
5378 Downstairs last five completed: 2209 2208 2207 2206 2205
5379 Upstairs last five completed: 2510 2509 2508 2507 2506
5380 CLIENT: Up:0 ds:301 act:2
5381 ----------------------------------------------------------------
5382 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5383 States: Offline Active Active
5384 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5385 2210 Acked Read 5 Sent Done Done false
5386 2211 Acked Read 10 Sent Done Done false
5387 2212 Acked Read 1 Sent Done Done false
5388 2213 Acked Write 6 Sent Done Done false
5389 2214 Acked Read 10 Sent Done Done false
5390 2215 Acked Read 7 Sent Done Done false
5391 2216 Acked Read 7 Sent Done Done false
5392 2217 Acked Read 5 Sent Done Done false
5393 2218 Acked Write 7 Sent Done Done false
5394 2219 Acked Write 3 Sent Done Done false
5395 2220 Acked Write 10 Sent Done Done false
5396 2221 Acked Read 5 Sent Done Done false
5397 2222 Acked Write 1 Sent Done Done false
5398 2223 Acked Write 5 Sent Done Done false
5399 2224 Acked Write 10 Sent Done Done false
5400 2225 Acked Read 6 Sent Done Done false
5401 2226 Acked Read 5 Sent Done Done false
5402 2227 Acked Write 10 Sent Done Done false
5403 2228 Acked Read 5 Sent Done Done false
5404 2229 Acked Write 1 Sent Done Done false
5405 2230 Acked Write 1 Sent Done Done false
5406 2231 Acked Write 7 Sent Done Done false
5407 2232 Acked Write 8 Sent Done Done false
5408 2233 Acked Read 9 Sent Done Done false
5409 2234 Acked Flush 0 Sent Done Done false
5410 2235 Acked Write 1 Sent Done Done false
5411 2236 Acked Write 4 Sent Done Done false
5412 2237 Acked Write 5 Sent Done Done false
5413 2238 Acked Read 8 Sent Done Done false
5414 2239 Acked Read 6 Sent Done Done false
5415 2240 Acked Write 7 Sent Done Done false
5416 2241 Acked Read 3 Sent Done Done false
5417 2242 Acked Write 9 Sent Done Done false
5418 2243 Acked Write 2 Sent Done Done false
5419 2244 Acked Write 1 Sent Done Done false
5420 2245 Acked Write 1 Sent Done Done false
5421 2246 Acked Write 7 Sent Done Done false
5422 2247 Acked Write 8 Sent Done Done false
5423 2248 Acked Read 1 Sent Done Done false
5424 2249 Acked Read 10 Sent Done Done false
5425 2250 Acked Write 6 Sent Done Done false
5426 2251 Acked Write 4 Sent Done Done false
5427 2252 Acked Write 2 Sent Done Done false
5428 2253 Acked Write 1 Sent Done Done false
5429 2254 Acked Read 4 Sent Done Done false
5430 2255 Acked Flush 0 Sent Done Done false
5431 2256 Acked Write 7 Sent Done Done false
5432 2257 Acked Write 9 Sent Done Done false
5433 2258 Acked Write 6 Sent Done Done false
5434 2259 Acked Write 4 Sent Done Done false
5435 2260 Acked Read 10 Sent Done Done false
5436 2261 Acked Write 9 Sent Done Done false
5437 2262 Acked Read 5 Sent Done Done false
5438 2263 Acked Read 4 Sent Done Done false
5439 2264 Acked Read 10 Sent Done Done false
5440 2265 Acked Read 1 Sent Done Done false
5441 2266 Acked Flush 0 Sent Done Done false
5442 2267 Acked Read 2 Sent Done Done false
5443 2268 Acked Flush 0 Sent Done Done false
5444 2269 Acked Write 8 Sent Done Done false
5445 2270 Acked Write 2 Sent Done Done false
5446 2271 Acked Write 8 Sent Done Done false
5447 2272 Acked Write 8 Sent Done Done false
5448 2273 Acked Flush 0 Sent Done Done false
5449 2274 Acked Read 6 Sent Done Done false
5450 2275 Acked Read 1 Sent Done Done false
5451 2276 Acked Read 4 Sent Done Done false
5452 2277 Acked Read 10 Sent Done Done false
5453 2278 Acked Read 6 Sent Done Done false
5454 2279 Acked Flush 0 Sent Done Done false
5455 2280 Acked Flush 0 Sent Done Done false
5456 2281 Acked Read 3 Sent Done Done false
5457 2282 Acked Write 3 Sent Done Done false
5458 2283 Acked Read 5 Sent Done Done false
5459 2284 Acked Flush 0 Sent Done Done false
5460 2285 Acked Read 10 Sent Done Done false
5461 2286 Acked Write 3 Sent Done Done false
5462 2287 Acked Read 7 Sent Done Done false
5463 2288 Acked Read 9 Sent Done Done false
5464 2289 Acked Write 7 Sent Done Done false
5465 2290 Acked Write 1 Sent Done Done false
5466 2291 Acked Write 2 Sent Done Done false
5467 2292 Acked Flush 0 Sent Done Done false
5468 2293 Acked Read 6 Sent Done Done false
5469 2294 Acked Read 3 Sent Done Done false
5470 2295 Acked Read 10 Sent Done Done false
5471 2296 Acked Flush 0 Sent Done Done false
5472 2297 Acked Write 2 Sent Done Done false
5473 2298 Acked Write 10 Sent Done Done false
5474 2299 Acked Write 6 Sent Done Done false
5475 2300 Acked Read 9 Sent Done Done false
5476 2301 Acked Read 9 Sent Done Done false
5477 2302 Acked Read 8 Sent Done Done false
5478 2303 Acked Read 9 Sent Done Done false
5479 2304 Acked Read 7 Sent Done Done false
5480 2305 Acked Read 6 Sent Done Done false
5481 2306 Acked Read 4 Sent Done Done false
5482 2307 Acked Read 9 Sent Done Done false
5483 2308 Acked Write 7 Sent Done Done false
5484 2309 Acked Read 6 Sent Done Done false
5485 2310 Acked Flush 0 Sent Done Done false
5486 2311 Acked Read 9 Sent Done Done false
5487 2312 Acked Write 1 Sent Done Done false
5488 2313 Acked Write 4 Sent Done Done false
5489 2314 Acked Read 6 Sent Done Done false
5490 2315 Acked Write 1 Sent Done Done false
5491 2316 Acked Read 2 Sent Done Done false
5492 2317 Acked Write 5 Sent Done Done false
5493 2318 Acked Read 4 Sent Done Done false
5494 2319 Acked Read 9 Sent Done Done false
5495 2320 Acked Write 4 Sent Done Done false
5496 2321 Acked Write 9 Sent Done Done false
5497 2322 Acked Read 3 Sent Done Done false
5498 2323 Acked Flush 0 Sent Done Done false
5499 2324 Acked Flush 0 Sent Done Done false
5500 2325 Acked Read 6 Sent Done Done false
5501 2326 Acked Read 10 Sent Done Done false
5502 2327 Acked Read 9 Sent Done Done false
5503 2328 Acked Read 8 Sent Done Done false
5504 2329 Acked Flush 0 Sent Done Done false
5505 2330 Acked Write 9 Sent Done Done false
5506 2331 Acked Flush 0 Sent Done Done false
5507 2332 Acked Read 6 Sent Done Done false
5508 2333 Acked Write 1 Sent Done Done false
5509 2334 Acked Read 3 Sent Done Done false
5510 2335 Acked Read 7 Sent Done Done false
5511 2336 Acked Read 3 Sent Done Done false
5512 2337 Acked Write 3 Sent Done Done false
5513 2338 Acked Write 7 Sent Done Done false
5514 2339 Acked Write 1 Sent Done Done false
5515 2340 Acked Read 3 Sent Done Done false
5516 2341 Acked Read 9 Sent Done Done false
5517 2342 Acked Write 9 Sent Done Done false
5518 2343 Acked Read 3 Sent Done Done false
5519 2344 Acked Read 5 Sent Done Done false
5520 2345 Acked Write 3 Sent Done Done false
5521 2346 Acked Read 9 Sent Done Done false
5522 2347 Acked Write 6 Sent Done Done false
5523 2348 Acked Read 9 Sent Done Done false
5524 2349 Acked Write 10 Sent Done Done false
5525 2350 Acked Read 6 Sent Done Done false
5526 2351 Acked Read 7 Sent Done Done false
5527 2352 Acked Write 7 Sent Done Done false
5528 2353 Acked Read 4 Sent Done Done false
5529 2354 Acked Flush 0 Sent Done Done false
5530 2355 Acked Write 10 Sent Done Done false
5531 2356 Acked Read 7 Sent Done Done false
5532 2357 Acked Write 7 Sent Done Done false
5533 2358 Acked Read 9 Sent Done Done false
5534 2359 Acked Read 4 Sent Done Done false
5535 2360 Acked Write 2 Sent Done Done false
5536 2361 Acked Write 8 Sent Done Done false
5537 2362 Acked Write 9 Sent Done Done false
5538 2363 Acked Read 7 Sent Done Done false
5539 2364 Acked Read 7 Sent Done Done false
5540 2365 Acked Write 7 Sent Done Done false
5541 2366 Acked Read 7 Sent Done Done false
5542 2367 Acked Flush 0 Sent Done Done false
5543 2368 Acked Read 5 Sent Done Done false
5544 2369 Acked Read 4 Sent Done Done false
5545 2370 Acked Read 2 Sent Done Done false
5546 2371 Acked Write 8 Sent Done Done false
5547 2372 Acked Read 9 Sent Done Done false
5548 2373 Acked Read 8 Sent Done Done false
5549 2374 Acked Read 10 Sent Done Done false
5550 2375 Acked Write 5 Sent Done Done false
5551 2376 Acked Read 6 Sent Done Done false
5552 2377 Acked Read 3 Sent Done Done false
5553 2378 Acked Flush 0 Sent Done Done false
5554 2379 Acked Write 9 Sent Done Done false
5555 2380 Acked Write 2 Sent Done Done false
5556 2381 Acked Flush 0 Sent Done Done false
5557 2382 Acked Write 3 Sent Done Done false
5558 2383 Acked Read 2 Sent Done Done false
5559 2384 Acked Write 3 Sent Done Done false
5560 2385 Acked Write 5 Sent Done Done false
5561 2386 Acked Write 1 Sent Done Done false
5562 2387 Acked Read 6 Sent Done Done false
5563 2388 Acked Write 7 Sent Done Done false
5564 2389 Acked Read 7 Sent Done Done false
5565 2390 Acked Flush 0 Sent Done Done false
5566 2391 Acked Read 5 Sent Done Done false
5567 2392 Acked Write 2 Sent Done Done false
5568 2393 Acked Read 2 Sent Done Done false
5569 2394 Acked Read 5 Sent Done Done false
5570 2395 Acked Read 9 Sent Done Done false
5571 2396 Acked Read 2 Sent Done Done false
5572 2397 Acked Read 1 Sent Done Done false
5573 2398 Acked Write 6 Sent Done Done false
5574 2399 Acked Read 10 Sent Done Done false
5575 2400 Acked Write 10 Sent Done Done false
5576 2401 Acked Write 7 Sent Done Done false
5577 2402 Acked Write 2 Sent Done Done false
5578 2403 Acked Read 2 Sent Done Done false
5579 2404 Acked Write 8 Sent Done Done false
5580 2405 Acked Write 4 Sent Done Done false
5581 2406 Acked Read 8 Sent Done Done false
5582 2407 Acked Read 5 Sent Done Done false
5583 2408 Acked Write 3 Sent Done Done false
5584 2409 Acked Read 7 Sent Done Done false
5585 2410 Acked Read 8 Sent Done Done false
5586 2411 Acked Read 1 Sent Done Done false
5587 2412 Acked Read 8 Sent Done Done false
5588 2413 Acked Write 1 Sent Done Done false
5589 2414 Acked Read 4 Sent Done Done false
5590 2415 Acked Read 10 Sent Done Done false
5591 2416 Acked Read 6 Sent Done Done false
5592 2417 Acked Write 5 Sent Done Done false
5593 2418 Acked Read 2 Sent Done Done false
5594 2419 Acked Read 8 Sent Done Done false
5595 2420 Acked Read 6 Sent Done Done false
5596 2421 Acked Flush 0 Sent Done Done false
5597 2422 Acked Read 10 Sent Done Done false
5598 2423 Acked Read 4 Sent Done Done false
5599 2424 Acked Read 6 Sent Done Done false
5600 2425 Acked Read 3 Sent Done Done false
5601 2426 Acked Read 4 Sent Done Done false
5602 2427 Acked Write 4 Sent Done Done false
5603 2428 Acked Read 10 Sent Done Done false
5604 2429 Acked Read 7 Sent Done Done false
5605 2430 Acked Write 4 Sent Done Done false
5606 2431 Acked Read 5 Sent Done Done false
5607 2432 Acked Write 5 Sent Done Done false
5608 2433 Acked Read 8 Sent Done Done false
5609 2434 Acked Read 9 Sent Done Done false
5610 2435 Acked Read 1 Sent Done Done false
5611 2436 Acked Write 10 Sent Done Done false
5612 2437 Acked Write 1 Sent Done Done false
5613 2438 Acked Write 2 Sent Done Done false
5614 2439 Acked Read 10 Sent Done Done false
5615 2440 Acked Read 10 Sent Done Done false
5616 2441 Acked Read 8 Sent Done Done false
5617 2442 Acked Read 8 Sent Done Done false
5618 2443 Acked Write 4 Sent Done Done false
5619 2444 Acked Write 2 Sent Done Done false
5620 2445 Acked Write 10 Sent Done Done false
5621 2446 Acked Read 1 Sent Done Done false
5622 2447 Acked Read 6 Sent Done Done false
5623 2448 Acked Read 2 Sent Done Done false
5624 2449 Acked Read 4 Sent Done Done false
5625 2450 Acked Read 1 Sent Done Done false
5626 2451 Acked Read 10 Sent Done Done false
5627 2452 Acked Write 3 Sent Done Done false
5628 2453 Acked Flush 0 Sent Done Done false
5629 2454 Acked Read 7 Sent Done Done false
5630 2455 Acked Write 7 Sent Done Done false
5631 2456 Acked Write 5 Sent Done Done false
5632 2457 Acked Read 6 Sent Done Done false
5633 2458 Acked Write 10 Sent Done Done false
5634 2459 Acked Write 7 Sent Done Done false
5635 2460 Acked Flush 0 Sent Done Done false
5636 2461 Acked Write 7 Sent Done Done false
5637 2462 Acked Read 6 Sent Done Done false
5638 2463 Acked Write 7 Sent Done Done false
5639 2464 Acked Write 10 Sent Done Done false
5640 2465 Acked Read 7 Sent Done Done false
5641 2466 Acked Write 3 Sent Done Done false
5642 2467 Acked Write 4 Sent Done Done false
5643 2468 Acked Read 4 Sent Done Done false
5644 2469 Acked Write 2 Sent Done Done false
5645 2470 Acked Read 5 Sent Done Done false
5646 2471 Acked Read 10 Sent Done Done false
5647 2472 Acked Read 4 Sent Done Done false
5648 2473 Acked Write 6 Sent Done Done false
5649 2474 Acked Read 6 Sent Done Done false
5650 2475 Acked Write 4 Sent Done Done false
5651 2476 Acked Flush 0 Sent Done Done false
5652 2477 Acked Write 9 Sent Done Done false
5653 2478 Acked Flush 0 Sent Done Done false
5654 2479 Acked Read 8 Sent Done Done false
5655 2480 Acked Read 4 Sent Done Done false
5656 2481 Acked Read 7 Sent Done Done false
5657 2482 Acked Write 5 Sent Done Done false
5658 2483 Acked Read 3 Sent Done Done false
5659 2484 Acked Write 9 Sent Done Done false
5660 2485 Acked Write 2 Sent Done Done false
5661 2486 Acked Read 7 Sent Done Done false
5662 2487 Acked Read 9 Sent Done Done false
5663 2488 Acked Write 3 Sent Done Done false
5664 2489 Acked Read 3 Sent Done Done false
5665 2490 Acked Read 6 Sent Done Done false
5666 2491 Acked Write 6 Sent Done Done false
5667 2492 Acked Read 5 Sent Done Done false
5668 2493 Acked Write 5 Sent Done Done false
5669 2494 Acked Write 4 Sent Done Done false
5670 2495 Acked Write 9 Sent Done Done false
5671 2496 Acked Read 4 Sent Done Done false
5672 2497 Acked Flush 0 Sent Done Done false
5673 2498 Acked Write 7 Sent Done Done false
5674 2499 Acked Read 10 Sent Done Done false
5675 2500 Acked Read 1 Sent Done Done false
5676 2501 Acked Read 4 Sent Done Done false
5677 2502 Acked Write 7 Sent Done Done false
5678 2503 Acked Read 8 Sent Done Done false
5679 2504 Acked Read 10 Sent Done Done false
5680 2505 Acked Write 1 Sent Done Done false
5681 2506 Acked Write 9 Sent Done Done false
5682 2507 Acked Write 9 Sent Done Done false
5683 2508 Acked Read 5 Sent Done Done false
5684 2509 Acked Write 10 Sent Done Done false
5685 2510 Acked Read 10 Sent Done Done false
5686 2511 Acked Flush 0 Sent Done Done false
5687 STATES DS:0 DS:1 DS:2 TOTAL
5688 Sent 302 0 0 302
5689 Done 0 302 302 604
5690 Skipped 0 0 0 0
5691 Error 0 0 0 0
5692 Last Flush: 2209 2511 2511
5693 Downstairs last five completed: 2209 2208 2207 2206 2205
5694 Upstairs last five completed: 2511 2510 2509 2508 2507
5695 CLIENT: Up:0 ds:302 act:2
56962024-11-07T14:04:28.978ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
56972024-11-07T14:04:28.978ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
56982024-11-07T14:04:28.978ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
56992024-11-07T14:04:28.978ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
57002024-11-07T14:04:28.978ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
5701 [8810] got start action
5702 [8810] I am going to restart
5703 Starting downstairs at port 8810
5704 Make output file at "/tmp/dsc/downstairs-8810.txt"
5705 [8810][0] reports Starting
5706 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1204)
5707 [8810][0] reports Running
5708 ----------------------------------------------------------------
5709 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5710 States: Offline Active Active
5711 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5712 2210 Acked Read 5 Sent Done Done false
5713 2211 Acked Read 10 Sent Done Done false
5714 2212 Acked Read 1 Sent Done Done false
5715 2213 Acked Write 6 Sent Done Done false
5716 2214 Acked Read 10 Sent Done Done false
5717 2215 Acked Read 7 Sent Done Done false
5718 2216 Acked Read 7 Sent Done Done false
5719 2217 Acked Read 5 Sent Done Done false
5720 2218 Acked Write 7 Sent Done Done false
5721 2219 Acked Write 3 Sent Done Done false
5722 2220 Acked Write 10 Sent Done Done false
5723 2221 Acked Read 5 Sent Done Done false
5724 2222 Acked Write 1 Sent Done Done false
5725 2223 Acked Write 5 Sent Done Done false
5726 2224 Acked Write 10 Sent Done Done false
5727 2225 Acked Read 6 Sent Done Done false
5728 2226 Acked Read 5 Sent Done Done false
5729 2227 Acked Write 10 Sent Done Done false
5730 2228 Acked Read 5 Sent Done Done false
5731 2229 Acked Write 1 Sent Done Done false
5732 2230 Acked Write 1 Sent Done Done false
5733 2231 Acked Write 7 Sent Done Done false
5734 2232 Acked Write 8 Sent Done Done false
5735 2233 Acked Read 9 Sent Done Done false
5736 2234 Acked Flush 0 Sent Done Done false
5737 2235 Acked Write 1 Sent Done Done false
5738 2236 Acked Write 4 Sent Done Done false
5739 2237 Acked Write 5 Sent Done Done false
5740 2238 Acked Read 8 Sent Done Done false
5741 2239 Acked Read 6 Sent Done Done false
5742 2240 Acked Write 7 Sent Done Done false
5743 2241 Acked Read 3 Sent Done Done false
5744 2242 Acked Write 9 Sent Done Done false
5745 2243 Acked Write 2 Sent Done Done false
5746 2244 Acked Write 1 Sent Done Done false
5747 2245 Acked Write 1 Sent Done Done false
5748 2246 Acked Write 7 Sent Done Done false
5749 2247 Acked Write 8 Sent Done Done false
5750 2248 Acked Read 1 Sent Done Done false
5751 2249 Acked Read 10 Sent Done Done false
5752 2250 Acked Write 6 Sent Done Done false
5753 2251 Acked Write 4 Sent Done Done false
5754 2252 Acked Write 2 Sent Done Done false
5755 2253 Acked Write 1 Sent Done Done false
5756 2254 Acked Read 4 Sent Done Done false
5757 2255 Acked Flush 0 Sent Done Done false
5758 2256 Acked Write 7 Sent Done Done false
5759 2257 Acked Write 9 Sent Done Done false
5760 2258 Acked Write 6 Sent Done Done false
5761 2259 Acked Write 4 Sent Done Done false
5762 2260 Acked Read 10 Sent Done Done false
5763 2261 Acked Write 9 Sent Done Done false
5764 2262 Acked Read 5 Sent Done Done false
5765 2263 Acked Read 4 Sent Done Done false
5766 2264 Acked Read 10 Sent Done Done false
5767 2265 Acked Read 1 Sent Done Done false
5768 2266 Acked Flush 0 Sent Done Done false
5769 2267 Acked Read 2 Sent Done Done false
5770 2268 Acked Flush 0 Sent Done Done false
5771 2269 Acked Write 8 Sent Done Done false
5772 2270 Acked Write 2 Sent Done Done false
5773 2271 Acked Write 8 Sent Done Done false
5774 2272 Acked Write 8 Sent Done Done false
5775 2273 Acked Flush 0 Sent Done Done false
5776 2274 Acked Read 6 Sent Done Done false
5777 2275 Acked Read 1 Sent Done Done false
5778 2276 Acked Read 4 Sent Done Done false
5779 2277 Acked Read 10 Sent Done Done false
5780 2278 Acked Read 6 Sent Done Done false
5781 2279 Acked Flush 0 Sent Done Done false
5782 2280 Acked Flush 0 Sent Done Done false
5783 2281 Acked Read 3 Sent Done Done false
5784 2282 Acked Write 3 Sent Done Done false
5785 2283 Acked Read 5 Sent Done Done false
5786 2284 Acked Flush 0 Sent Done Done false
5787 2285 Acked Read 10 Sent Done Done false
5788 2286 Acked Write 3 Sent Done Done false
5789 2287 Acked Read 7 Sent Done Done false
5790 2288 Acked Read 9 Sent Done Done false
5791 2289 Acked Write 7 Sent Done Done false
5792 2290 Acked Write 1 Sent Done Done false
5793 2291 Acked Write 2 Sent Done Done false
5794 2292 Acked Flush 0 Sent Done Done false
5795 2293 Acked Read 6 Sent Done Done false
5796 2294 Acked Read 3 Sent Done Done false
5797 2295 Acked Read 10 Sent Done Done false
5798 2296 Acked Flush 0 Sent Done Done false
5799 2297 Acked Write 2 Sent Done Done false
5800 2298 Acked Write 10 Sent Done Done false
5801 2299 Acked Write 6 Sent Done Done false
5802 2300 Acked Read 9 Sent Done Done false
5803 2301 Acked Read 9 Sent Done Done false
5804 2302 Acked Read 8 Sent Done Done false
5805 2303 Acked Read 9 Sent Done Done false
5806 2304 Acked Read 7 Sent Done Done false
5807 2305 Acked Read 6 Sent Done Done false
5808 2306 Acked Read 4 Sent Done Done false
5809 2307 Acked Read 9 Sent Done Done false
5810 2308 Acked Write 7 Sent Done Done false
5811 2309 Acked Read 6 Sent Done Done false
5812 2310 Acked Flush 0 Sent Done Done false
5813 2311 Acked Read 9 Sent Done Done false
5814 2312 Acked Write 1 Sent Done Done false
5815 2313 Acked Write 4 Sent Done Done false
5816 2314 Acked Read 6 Sent Done Done false
5817 2315 Acked Write 1 Sent Done Done false
5818 2316 Acked Read 2 Sent Done Done false
5819 2317 Acked Write 5 Sent Done Done false
5820 2318 Acked Read 4 Sent Done Done false
5821 2319 Acked Read 9 Sent Done Done false
5822 2320 Acked Write 4 Sent Done Done false
5823 2321 Acked Write 9 Sent Done Done false
5824 2322 Acked Read 3 Sent Done Done false
5825 2323 Acked Flush 0 Sent Done Done false
5826 2324 Acked Flush 0 Sent Done Done false
5827 2325 Acked Read 6 Sent Done Done false
5828 2326 Acked Read 10 Sent Done Done false
5829 2327 Acked Read 9 Sent Done Done false
5830 2328 Acked Read 8 Sent Done Done false
5831 2329 Acked Flush 0 Sent Done Done false
5832 2330 Acked Write 9 Sent Done Done false
5833 2331 Acked Flush 0 Sent Done Done false
5834 2332 Acked Read 6 Sent Done Done false
5835 2333 Acked Write 1 Sent Done Done false
5836 2334 Acked Read 3 Sent Done Done false
5837 2335 Acked Read 7 Sent Done Done false
5838 2336 Acked Read 3 Sent Done Done false
5839 2337 Acked Write 3 Sent Done Done false
5840 2338 Acked Write 7 Sent Done Done false
5841 2339 Acked Write 1 Sent Done Done false
5842 2340 Acked Read 3 Sent Done Done false
5843 2341 Acked Read 9 Sent Done Done false
5844 2342 Acked Write 9 Sent Done Done false
5845 2343 Acked Read 3 Sent Done Done false
5846 2344 Acked Read 5 Sent Done Done false
5847 2345 Acked Write 3 Sent Done Done false
5848 2346 Acked Read 9 Sent Done Done false
5849 2347 Acked Write 6 Sent Done Done false
5850 2348 Acked Read 9 Sent Done Done false
5851 2349 Acked Write 10 Sent Done Done false
5852 2350 Acked Read 6 Sent Done Done false
5853 2351 Acked Read 7 Sent Done Done false
5854 2352 Acked Write 7 Sent Done Done false
5855 2353 Acked Read 4 Sent Done Done false
5856 2354 Acked Flush 0 Sent Done Done false
5857 2355 Acked Write 10 Sent Done Done false
5858 2356 Acked Read 7 Sent Done Done false
5859 2357 Acked Write 7 Sent Done Done false
5860 2358 Acked Read 9 Sent Done Done false
5861 2359 Acked Read 4 Sent Done Done false
5862 2360 Acked Write 2 Sent Done Done false
5863 2361 Acked Write 8 Sent Done Done false
5864 2362 Acked Write 9 Sent Done Done false
5865 2363 Acked Read 7 Sent Done Done false
5866 2364 Acked Read 7 Sent Done Done false
5867 2365 Acked Write 7 Sent Done Done false
5868 2366 Acked Read 7 Sent Done Done false
5869 2367 Acked Flush 0 Sent Done Done false
5870 2368 Acked Read 5 Sent Done Done false
5871 2369 Acked Read 4 Sent Done Done false
5872 2370 Acked Read 2 Sent Done Done false
5873 2371 Acked Write 8 Sent Done Done false
5874 2372 Acked Read 9 Sent Done Done false
5875 2373 Acked Read 8 Sent Done Done false
5876 2374 Acked Read 10 Sent Done Done false
5877 2375 Acked Write 5 Sent Done Done false
5878 2376 Acked Read 6 Sent Done Done false
5879 2377 Acked Read 3 Sent Done Done false
5880 2378 Acked Flush 0 Sent Done Done false
5881 2379 Acked Write 9 Sent Done Done false
5882 2380 Acked Write 2 Sent Done Done false
5883 2381 Acked Flush 0 Sent Done Done false
5884 2382 Acked Write 3 Sent Done Done false
5885 2383 Acked Read 2 Sent Done Done false
5886 2384 Acked Write 3 Sent Done Done false
5887 2385 Acked Write 5 Sent Done Done false
5888 2386 Acked Write 1 Sent Done Done false
5889 2387 Acked Read 6 Sent Done Done false
5890 2388 Acked Write 7 Sent Done Done false
5891 2389 Acked Read 7 Sent Done Done false
5892 2390 Acked Flush 0 Sent Done Done false
5893 2391 Acked Read 5 Sent Done Done false
5894 2392 Acked Write 2 Sent Done Done false
5895 2393 Acked Read 2 Sent Done Done false
5896 2394 Acked Read 5 Sent Done Done false
5897 2395 Acked Read 9 Sent Done Done false
5898 2396 Acked Read 2 Sent Done Done false
5899 2397 Acked Read 1 Sent Done Done false
5900 2398 Acked Write 6 Sent Done Done false
5901 2399 Acked Read 10 Sent Done Done false
5902 2400 Acked Write 10 Sent Done Done false
5903 2401 Acked Write 7 Sent Done Done false
5904 2402 Acked Write 2 Sent Done Done false
5905 2403 Acked Read 2 Sent Done Done false
5906 2404 Acked Write 8 Sent Done Done false
5907 2405 Acked Write 4 Sent Done Done false
5908 2406 Acked Read 8 Sent Done Done false
5909 2407 Acked Read 5 Sent Done Done false
5910 2408 Acked Write 3 Sent Done Done false
5911 2409 Acked Read 7 Sent Done Done false
5912 2410 Acked Read 8 Sent Done Done false
5913 2411 Acked Read 1 Sent Done Done false
5914 2412 Acked Read 8 Sent Done Done false
5915 2413 Acked Write 1 Sent Done Done false
5916 2414 Acked Read 4 Sent Done Done false
5917 2415 Acked Read 10 Sent Done Done false
5918 2416 Acked Read 6 Sent Done Done false
5919 2417 Acked Write 5 Sent Done Done false
5920 2418 Acked Read 2 Sent Done Done false
5921 2419 Acked Read 8 Sent Done Done false
5922 2420 Acked Read 6 Sent Done Done false
5923 2421 Acked Flush 0 Sent Done Done false
5924 2422 Acked Read 10 Sent Done Done false
5925 2423 Acked Read 4 Sent Done Done false
5926 2424 Acked Read 6 Sent Done Done false
5927 2425 Acked Read 3 Sent Done Done false
5928 2426 Acked Read 4 Sent Done Done false
5929 2427 Acked Write 4 Sent Done Done false
5930 2428 Acked Read 10 Sent Done Done false
5931 2429 Acked Read 7 Sent Done Done false
5932 2430 Acked Write 4 Sent Done Done false
5933 2431 Acked Read 5 Sent Done Done false
5934 2432 Acked Write 5 Sent Done Done false
5935 2433 Acked Read 8 Sent Done Done false
5936 2434 Acked Read 9 Sent Done Done false
5937 2435 Acked Read 1 Sent Done Done false
5938 2436 Acked Write 10 Sent Done Done false
5939 2437 Acked Write 1 Sent Done Done false
5940 2438 Acked Write 2 Sent Done Done false
5941 2439 Acked Read 10 Sent Done Done false
5942 2440 Acked Read 10 Sent Done Done false
5943 2441 Acked Read 8 Sent Done Done false
5944 2442 Acked Read 8 Sent Done Done false
5945 2443 Acked Write 4 Sent Done Done false
5946 2444 Acked Write 2 Sent Done Done false
5947 2445 Acked Write 10 Sent Done Done false
5948 2446 Acked Read 1 Sent Done Done false
5949 2447 Acked Read 6 Sent Done Done false
5950 2448 Acked Read 2 Sent Done Done false
5951 2449 Acked Read 4 Sent Done Done false
5952 2450 Acked Read 1 Sent Done Done false
5953 2451 Acked Read 10 Sent Done Done false
5954 2452 Acked Write 3 Sent Done Done false
5955 2453 Acked Flush 0 Sent Done Done false
5956 2454 Acked Read 7 Sent Done Done false
5957 2455 Acked Write 7 Sent Done Done false
5958 2456 Acked Write 5 Sent Done Done false
5959 2457 Acked Read 6 Sent Done Done false
5960 2458 Acked Write 10 Sent Done Done false
5961 2459 Acked Write 7 Sent Done Done false
5962 2460 Acked Flush 0 Sent Done Done false
5963 2461 Acked Write 7 Sent Done Done false
5964 2462 Acked Read 6 Sent Done Done false
5965 2463 Acked Write 7 Sent Done Done false
5966 2464 Acked Write 10 Sent Done Done false
5967 2465 Acked Read 7 Sent Done Done false
5968 2466 Acked Write 3 Sent Done Done false
5969 2467 Acked Write 4 Sent Done Done false
5970 2468 Acked Read 4 Sent Done Done false
5971 2469 Acked Write 2 Sent Done Done false
5972 2470 Acked Read 5 Sent Done Done false
5973 2471 Acked Read 10 Sent Done Done false
5974 2472 Acked Read 4 Sent Done Done false
5975 2473 Acked Write 6 Sent Done Done false
5976 2474 Acked Read 6 Sent Done Done false
5977 2475 Acked Write 4 Sent Done Done false
5978 2476 Acked Flush 0 Sent Done Done false
5979 2477 Acked Write 9 Sent Done Done false
5980 2478 Acked Flush 0 Sent Done Done false
5981 2479 Acked Read 8 Sent Done Done false
5982 2480 Acked Read 4 Sent Done Done false
5983 2481 Acked Read 7 Sent Done Done false
5984 2482 Acked Write 5 Sent Done Done false
5985 2483 Acked Read 3 Sent Done Done false
5986 2484 Acked Write 9 Sent Done Done false
5987 2485 Acked Write 2 Sent Done Done false
5988 2486 Acked Read 7 Sent Done Done false
5989 2487 Acked Read 9 Sent Done Done false
5990 2488 Acked Write 3 Sent Done Done false
5991 2489 Acked Read 3 Sent Done Done false
5992 2490 Acked Read 6 Sent Done Done false
5993 2491 Acked Write 6 Sent Done Done false
5994 2492 Acked Read 5 Sent Done Done false
5995 2493 Acked Write 5 Sent Done Done false
5996 2494 Acked Write 4 Sent Done Done false
5997 2495 Acked Write 9 Sent Done Done false
5998 2496 Acked Read 4 Sent Done Done false
5999 2497 Acked Flush 0 Sent Done Done false
6000 2498 Acked Write 7 Sent Done Done false
6001 2499 Acked Read 10 Sent Done Done false
6002 2500 Acked Read 1 Sent Done Done false
6003 2501 Acked Read 4 Sent Done Done false
6004 2502 Acked Write 7 Sent Done Done false
6005 2503 Acked Read 8 Sent Done Done false
6006 2504 Acked Read 10 Sent Done Done false
6007 2505 Acked Write 1 Sent Done Done false
6008 2506 Acked Write 9 Sent Done Done false
6009 2507 Acked Write 9 Sent Done Done false
6010 2508 Acked Read 5 Sent Done Done false
6011 2509 Acked Write 10 Sent Done Done false
6012 2510 Acked Read 10 Sent Done Done false
6013 2511 Acked Flush 0 Sent Done Done false
6014 STATES DS:0 DS:1 DS:2 TOTAL
6015 Sent 302 0 0 302
6016 Done 0 302 302 604
6017 Skipped 0 0 0 0
6018 Error 0 0 0 0
6019 Last Flush: 2209 2511 2511
6020 Downstairs last five completed: 2209 2208 2207 2206 2205
6021 Upstairs last five completed: 2511 2510 2509 2508 2507
6022 CLIENT: Up:0 ds:302 act:2
6023 ----------------------------------------------------------------
6024 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6025 States: Offline Active Active
6026 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6027 2210 Acked Read 5 Sent Done Done false
6028 2211 Acked Read 10 Sent Done Done false
6029 2212 Acked Read 1 Sent Done Done false
6030 2213 Acked Write 6 Sent Done Done false
6031 2214 Acked Read 10 Sent Done Done false
6032 2215 Acked Read 7 Sent Done Done false
6033 2216 Acked Read 7 Sent Done Done false
6034 2217 Acked Read 5 Sent Done Done false
6035 2218 Acked Write 7 Sent Done Done false
6036 2219 Acked Write 3 Sent Done Done false
6037 2220 Acked Write 10 Sent Done Done false
6038 2221 Acked Read 5 Sent Done Done false
6039 2222 Acked Write 1 Sent Done Done false
6040 2223 Acked Write 5 Sent Done Done false
6041 2224 Acked Write 10 Sent Done Done false
6042 2225 Acked Read 6 Sent Done Done false
6043 2226 Acked Read 5 Sent Done Done false
6044 2227 Acked Write 10 Sent Done Done false
6045 2228 Acked Read 5 Sent Done Done false
6046 2229 Acked Write 1 Sent Done Done false
6047 2230 Acked Write 1 Sent Done Done false
6048 2231 Acked Write 7 Sent Done Done false
6049 2232 Acked Write 8 Sent Done Done false
6050 2233 Acked Read 9 Sent Done Done false
6051 2234 Acked Flush 0 Sent Done Done false
6052 2235 Acked Write 1 Sent Done Done false
6053 2236 Acked Write 4 Sent Done Done false
6054 2237 Acked Write 5 Sent Done Done false
6055 2238 Acked Read 8 Sent Done Done false
6056 2239 Acked Read 6 Sent Done Done false
6057 2240 Acked Write 7 Sent Done Done false
6058 2241 Acked Read 3 Sent Done Done false
6059 2242 Acked Write 9 Sent Done Done false
6060 2243 Acked Write 2 Sent Done Done false
6061 2244 Acked Write 1 Sent Done Done false
6062 2245 Acked Write 1 Sent Done Done false
6063 2246 Acked Write 7 Sent Done Done false
6064 2247 Acked Write 8 Sent Done Done false
6065 2248 Acked Read 1 Sent Done Done false
6066 2249 Acked Read 10 Sent Done Done false
6067 2250 Acked Write 6 Sent Done Done false
6068 2251 Acked Write 4 Sent Done Done false
6069 2252 Acked Write 2 Sent Done Done false
6070 2253 Acked Write 1 Sent Done Done false
6071 2254 Acked Read 4 Sent Done Done false
6072 2255 Acked Flush 0 Sent Done Done false
6073 2256 Acked Write 7 Sent Done Done false
6074 2257 Acked Write 9 Sent Done Done false
6075 2258 Acked Write 6 Sent Done Done false
6076 2259 Acked Write 4 Sent Done Done false
6077 2260 Acked Read 10 Sent Done Done false
6078 2261 Acked Write 9 Sent Done Done false
6079 2262 Acked Read 5 Sent Done Done false
6080 2263 Acked Read 4 Sent Done Done false
6081 2264 Acked Read 10 Sent Done Done false
6082 2265 Acked Read 1 Sent Done Done false
6083 2266 Acked Flush 0 Sent Done Done false
6084 2267 Acked Read 2 Sent Done Done false
6085 2268 Acked Flush 0 Sent Done Done false
6086 2269 Acked Write 8 Sent Done Done false
6087 2270 Acked Write 2 Sent Done Done false
6088 2271 Acked Write 8 Sent Done Done false
6089 2272 Acked Write 8 Sent Done Done false
6090 2273 Acked Flush 0 Sent Done Done false
6091 2274 Acked Read 6 Sent Done Done false
6092 2275 Acked Read 1 Sent Done Done false
6093 2276 Acked Read 4 Sent Done Done false
6094 2277 Acked Read 10 Sent Done Done false
6095 2278 Acked Read 6 Sent Done Done false
6096 2279 Acked Flush 0 Sent Done Done false
6097 2280 Acked Flush 0 Sent Done Done false
6098 2281 Acked Read 3 Sent Done Done false
6099 2282 Acked Write 3 Sent Done Done false
6100 2283 Acked Read 5 Sent Done Done false
6101 2284 Acked Flush 0 Sent Done Done false
6102 2285 Acked Read 10 Sent Done Done false
6103 2286 Acked Write 3 Sent Done Done false
6104 2287 Acked Read 7 Sent Done Done false
6105 2288 Acked Read 9 Sent Done Done false
6106 2289 Acked Write 7 Sent Done Done false
6107 2290 Acked Write 1 Sent Done Done false
6108 2291 Acked Write 2 Sent Done Done false
6109 2292 Acked Flush 0 Sent Done Done false
6110 2293 Acked Read 6 Sent Done Done false
6111 2294 Acked Read 3 Sent Done Done false
6112 2295 Acked Read 10 Sent Done Done false
6113 2296 Acked Flush 0 Sent Done Done false
6114 2297 Acked Write 2 Sent Done Done false
6115 2298 Acked Write 10 Sent Done Done false
6116 2299 Acked Write 6 Sent Done Done false
6117 2300 Acked Read 9 Sent Done Done false
6118 2301 Acked Read 9 Sent Done Done false
6119 2302 Acked Read 8 Sent Done Done false
6120 2303 Acked Read 9 Sent Done Done false
6121 2304 Acked Read 7 Sent Done Done false
6122 2305 Acked Read 6 Sent Done Done false
6123 2306 Acked Read 4 Sent Done Done false
6124 2307 Acked Read 9 Sent Done Done false
6125 2308 Acked Write 7 Sent Done Done false
6126 2309 Acked Read 6 Sent Done Done false
6127 2310 Acked Flush 0 Sent Done Done false
6128 2311 Acked Read 9 Sent Done Done false
6129 2312 Acked Write 1 Sent Done Done false
6130 2313 Acked Write 4 Sent Done Done false
6131 2314 Acked Read 6 Sent Done Done false
6132 2315 Acked Write 1 Sent Done Done false
6133 2316 Acked Read 2 Sent Done Done false
6134 2317 Acked Write 5 Sent Done Done false
6135 2318 Acked Read 4 Sent Done Done false
6136 2319 Acked Read 9 Sent Done Done false
6137 2320 Acked Write 4 Sent Done Done false
6138 2321 Acked Write 9 Sent Done Done false
6139 2322 Acked Read 3 Sent Done Done false
6140 2323 Acked Flush 0 Sent Done Done false
6141 2324 Acked Flush 0 Sent Done Done false
6142 2325 Acked Read 6 Sent Done Done false
6143 2326 Acked Read 10 Sent Done Done false
6144 2327 Acked Read 9 Sent Done Done false
6145 2328 Acked Read 8 Sent Done Done false
6146 2329 Acked Flush 0 Sent Done Done false
6147 2330 Acked Write 9 Sent Done Done false
6148 2331 Acked Flush 0 Sent Done Done false
6149 2332 Acked Read 6 Sent Done Done false
6150 2333 Acked Write 1 Sent Done Done false
6151 2334 Acked Read 3 Sent Done Done false
6152 2335 Acked Read 7 Sent Done Done false
6153 2336 Acked Read 3 Sent Done Done false
6154 2337 Acked Write 3 Sent Done Done false
6155 2338 Acked Write 7 Sent Done Done false
6156 2339 Acked Write 1 Sent Done Done false
6157 2340 Acked Read 3 Sent Done Done false
6158 2341 Acked Read 9 Sent Done Done false
6159 2342 Acked Write 9 Sent Done Done false
6160 2343 Acked Read 3 Sent Done Done false
6161 2344 Acked Read 5 Sent Done Done false
6162 2345 Acked Write 3 Sent Done Done false
6163 2346 Acked Read 9 Sent Done Done false
6164 2347 Acked Write 6 Sent Done Done false
6165 2348 Acked Read 9 Sent Done Done false
6166 2349 Acked Write 10 Sent Done Done false
6167 2350 Acked Read 6 Sent Done Done false
6168 2351 Acked Read 7 Sent Done Done false
6169 2352 Acked Write 7 Sent Done Done false
6170 2353 Acked Read 4 Sent Done Done false
6171 2354 Acked Flush 0 Sent Done Done false
6172 2355 Acked Write 10 Sent Done Done false
6173 2356 Acked Read 7 Sent Done Done false
6174 2357 Acked Write 7 Sent Done Done false
6175 2358 Acked Read 9 Sent Done Done false
6176 2359 Acked Read 4 Sent Done Done false
6177 2360 Acked Write 2 Sent Done Done false
6178 2361 Acked Write 8 Sent Done Done false
6179 2362 Acked Write 9 Sent Done Done false
6180 2363 Acked Read 7 Sent Done Done false
6181 2364 Acked Read 7 Sent Done Done false
6182 2365 Acked Write 7 Sent Done Done false
6183 2366 Acked Read 7 Sent Done Done false
6184 2367 Acked Flush 0 Sent Done Done false
6185 2368 Acked Read 5 Sent Done Done false
6186 2369 Acked Read 4 Sent Done Done false
6187 2370 Acked Read 2 Sent Done Done false
6188 2371 Acked Write 8 Sent Done Done false
6189 2372 Acked Read 9 Sent Done Done false
6190 2373 Acked Read 8 Sent Done Done false
6191 2374 Acked Read 10 Sent Done Done false
6192 2375 Acked Write 5 Sent Done Done false
6193 2376 Acked Read 6 Sent Done Done false
6194 2377 Acked Read 3 Sent Done Done false
6195 2378 Acked Flush 0 Sent Done Done false
6196 2379 Acked Write 9 Sent Done Done false
6197 2380 Acked Write 2 Sent Done Done false
6198 2381 Acked Flush 0 Sent Done Done false
6199 2382 Acked Write 3 Sent Done Done false
6200 2383 Acked Read 2 Sent Done Done false
6201 2384 Acked Write 3 Sent Done Done false
6202 2385 Acked Write 5 Sent Done Done false
6203 2386 Acked Write 1 Sent Done Done false
6204 2387 Acked Read 6 Sent Done Done false
6205 2388 Acked Write 7 Sent Done Done false
6206 2389 Acked Read 7 Sent Done Done false
6207 2390 Acked Flush 0 Sent Done Done false
6208 2391 Acked Read 5 Sent Done Done false
6209 2392 Acked Write 2 Sent Done Done false
6210 2393 Acked Read 2 Sent Done Done false
6211 2394 Acked Read 5 Sent Done Done false
6212 2395 Acked Read 9 Sent Done Done false
6213 2396 Acked Read 2 Sent Done Done false
6214 2397 Acked Read 1 Sent Done Done false
6215 2398 Acked Write 6 Sent Done Done false
6216 2399 Acked Read 10 Sent Done Done false
6217 2400 Acked Write 10 Sent Done Done false
6218 2401 Acked Write 7 Sent Done Done false
6219 2402 Acked Write 2 Sent Done Done false
6220 2403 Acked Read 2 Sent Done Done false
6221 2404 Acked Write 8 Sent Done Done false
6222 2405 Acked Write 4 Sent Done Done false
6223 2406 Acked Read 8 Sent Done Done false
6224 2407 Acked Read 5 Sent Done Done false
6225 2408 Acked Write 3 Sent Done Done false
6226 2409 Acked Read 7 Sent Done Done false
6227 2410 Acked Read 8 Sent Done Done false
6228 2411 Acked Read 1 Sent Done Done false
6229 2412 Acked Read 8 Sent Done Done false
6230 2413 Acked Write 1 Sent Done Done false
6231 2414 Acked Read 4 Sent Done Done false
6232 2415 Acked Read 10 Sent Done Done false
6233 2416 Acked Read 6 Sent Done Done false
6234 2417 Acked Write 5 Sent Done Done false
6235 2418 Acked Read 2 Sent Done Done false
6236 2419 Acked Read 8 Sent Done Done false
6237 2420 Acked Read 6 Sent Done Done false
6238 2421 Acked Flush 0 Sent Done Done false
6239 2422 Acked Read 10 Sent Done Done false
6240 2423 Acked Read 4 Sent Done Done false
6241 2424 Acked Read 6 Sent Done Done false
6242 2425 Acked Read 3 Sent Done Done false
6243 2426 Acked Read 4 Sent Done Done false
6244 2427 Acked Write 4 Sent Done Done false
6245 2428 Acked Read 10 Sent Done Done false
6246 2429 Acked Read 7 Sent Done Done false
6247 2430 Acked Write 4 Sent Done Done false
6248 2431 Acked Read 5 Sent Done Done false
6249 2432 Acked Write 5 Sent Done Done false
6250 2433 Acked Read 8 Sent Done Done false
6251 2434 Acked Read 9 Sent Done Done false
6252 2435 Acked Read 1 Sent Done Done false
6253 2436 Acked Write 10 Sent Done Done false
6254 2437 Acked Write 1 Sent Done Done false
6255 2438 Acked Write 2 Sent Done Done false
6256 2439 Acked Read 10 Sent Done Done false
6257 2440 Acked Read 10 Sent Done Done false
6258 2441 Acked Read 8 Sent Done Done false
6259 2442 Acked Read 8 Sent Done Done false
6260 2443 Acked Write 4 Sent Done Done false
6261 2444 Acked Write 2 Sent Done Done false
6262 2445 Acked Write 10 Sent Done Done false
6263 2446 Acked Read 1 Sent Done Done false
6264 2447 Acked Read 6 Sent Done Done false
6265 2448 Acked Read 2 Sent Done Done false
6266 2449 Acked Read 4 Sent Done Done false
6267 2450 Acked Read 1 Sent Done Done false
6268 2451 Acked Read 10 Sent Done Done false
6269 2452 Acked Write 3 Sent Done Done false
6270 2453 Acked Flush 0 Sent Done Done false
6271 2454 Acked Read 7 Sent Done Done false
6272 2455 Acked Write 7 Sent Done Done false
6273 2456 Acked Write 5 Sent Done Done false
6274 2457 Acked Read 6 Sent Done Done false
6275 2458 Acked Write 10 Sent Done Done false
6276 2459 Acked Write 7 Sent Done Done false
6277 2460 Acked Flush 0 Sent Done Done false
6278 2461 Acked Write 7 Sent Done Done false
6279 2462 Acked Read 6 Sent Done Done false
6280 2463 Acked Write 7 Sent Done Done false
6281 2464 Acked Write 10 Sent Done Done false
6282 2465 Acked Read 7 Sent Done Done false
6283 2466 Acked Write 3 Sent Done Done false
6284 2467 Acked Write 4 Sent Done Done false
6285 2468 Acked Read 4 Sent Done Done false
6286 2469 Acked Write 2 Sent Done Done false
6287 2470 Acked Read 5 Sent Done Done false
6288 2471 Acked Read 10 Sent Done Done false
6289 2472 Acked Read 4 Sent Done Done false
6290 2473 Acked Write 6 Sent Done Done false
6291 2474 Acked Read 6 Sent Done Done false
6292 2475 Acked Write 4 Sent Done Done false
6293 2476 Acked Flush 0 Sent Done Done false
6294 2477 Acked Write 9 Sent Done Done false
6295 2478 Acked Flush 0 Sent Done Done false
6296 2479 Acked Read 8 Sent Done Done false
6297 2480 Acked Read 4 Sent Done Done false
6298 2481 Acked Read 7 Sent Done Done false
6299 2482 Acked Write 5 Sent Done Done false
6300 2483 Acked Read 3 Sent Done Done false
6301 2484 Acked Write 9 Sent Done Done false
6302 2485 Acked Write 2 Sent Done Done false
6303 2486 Acked Read 7 Sent Done Done false
6304 2487 Acked Read 9 Sent Done Done false
6305 2488 Acked Write 3 Sent Done Done false
6306 2489 Acked Read 3 Sent Done Done false
6307 2490 Acked Read 6 Sent Done Done false
6308 2491 Acked Write 6 Sent Done Done false
6309 2492 Acked Read 5 Sent Done Done false
6310 2493 Acked Write 5 Sent Done Done false
6311 2494 Acked Write 4 Sent Done Done false
6312 2495 Acked Write 9 Sent Done Done false
6313 2496 Acked Read 4 Sent Done Done false
6314 2497 Acked Flush 0 Sent Done Done false
6315 2498 Acked Write 7 Sent Done Done false
6316 2499 Acked Read 10 Sent Done Done false
6317 2500 Acked Read 1 Sent Done Done false
6318 2501 Acked Read 4 Sent Done Done false
6319 2502 Acked Write 7 Sent Done Done false
6320 2503 Acked Read 8 Sent Done Done false
6321 2504 Acked Read 10 Sent Done Done false
6322 2505 Acked Write 1 Sent Done Done false
6323 2506 Acked Write 9 Sent Done Done false
6324 2507 Acked Write 9 Sent Done Done false
6325 2508 Acked Read 5 Sent Done Done false
6326 2509 Acked Write 10 Sent Done Done false
6327 2510 Acked Read 10 Sent Done Done false
6328 2511 Acked Flush 0 Sent Done Done false
6329 STATES DS:0 DS:1 DS:2 TOTAL
6330 Sent 302 0 0 302
6331 Done 0 302 302 604
6332 Skipped 0 0 0 0
6333 Error 0 0 0 0
6334 Last Flush: 2209 2511 2511
6335 Downstairs last five completed: 2209 2208 2207 2206 2205
6336 Upstairs last five completed: 2511 2510 2509 2508 2507
6337 CLIENT: Up:0 ds:302 act:2
63382024-11-07T14:04:38.979ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63392024-11-07T14:04:38.979ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58074) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63402024-11-07T14:04:38.980ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63412024-11-07T14:04:38.981ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63422024-11-07T14:04:38.981ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63432024-11-07T14:04:38.981ZINFOcrucible: send last flush ID to this DS: 2209 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63442024-11-07T14:04:38.981ZINFOcrucible: Replied this last flush ID: 2209 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63452024-11-07T14:04:38.981ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63462024-11-07T14:04:38.981ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63472024-11-07T14:04:38.981ZINFOcrucible: [0] client re-new 302 jobs since flush 2209 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63482024-11-07T14:04:38.981ZINFOcrucible: [0] Replayed 302 jobs since flush: 2209 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
6349 ----------------------------------------------------------------
6350 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6351 Downstairs last five completed: 2511 2510 2509 2508 2507
6352 Upstairs last five completed: 2511 2510 2509 2508 2507
6353 CLIENT: Up:0 ds:0 act:3
6354 Replay: All jobs finished
6355 Main task has work to do, go find it
6356 got dsc Stop(1)
6357 stop 1
6358 Nov 07 14:04:39.276 INFO request completed, latency_us: 381, response_code: 204, uri: /stop/cid/1[8820] Got stop action so:false kr:false
6359 , method: POST, req_id: 2e560f57-8d75-43c2-a77f-d3da99542f09, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
6360 Nov 07 14:04:39.276 INFO request completed, latency_us: 136, response_code: 200, uri: /state/cid/1, method: GET, req_id: 45e8488b-c963-4511-a3e1-6b127a79fde0, remote_addr: 127.0.0.1:34100, local_addr{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-07T14:04:39.276813187Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"":"io task","client":"1","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
6361 : 127.0.0.1:9998
63622024-11-07T14:04:39.276ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63632024-11-07T14:04:39.276ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63642024-11-07T14:04:39.276ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63652024-11-07T14:04:39.276ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
63662024-11-07T14:04:39.277ZINFOcrucible: client task is exiting = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
6367 [8820] Exited with: ExitStatus(unix_wait_status(9))
6368 [8820][1] reports Exit
6369 Nov 07 14:04:43.279 INFO request completed, latency_us: 277, response_code: 200, uri: /state/cid/1, method: GET, req_id: d11d49cf-3dfa-4909-8d68-47cb7a0cf04f, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
6370 001/300 Write block 575 len 28672 data: 3 3 2 2 2 3 3
6371 002/300 Read block 495 len 8192
6372 003/300 Read block 429 len 12288
6373 004/300 Read block 3362 len 4096
6374 005/300 Flush
6375 006/300 Write block 2043 len 24576 data: 2 2 2 2 2 3
6376 007/300 Write block 791 len 28672 data: 3 3 3 3 3 3 2
6377 008/300 Read block 421 len 36864
6378 009/300 Read block 1704 len 32768
6379 010/300 Write block 4774 len 32768 data: 3 2 2 2 2 2 2 2
6380 011/300 Write block 527 len 12288 data: 3 3 3
6381 012/300 Write block 4044 len 8192 data: 3 3
6382 013/300 Write block 4651 len 4096 data: 2
6383 014/300 Write block 4521 len 32768 data: 3 3 3 3 3 3 3 3
6384 015/300 Write block 1015 len 4096 data: 3
6385 016/300 Read block 1593 len 4096
6386 017/300 Flush
6387 018/300 Read block 35 len 32768
6388 019/300 Flush
6389 020/300 Read block 300 len 32768
6390 021/300 Flush
6391 022/300 Write block 81 len 36864 data: 2 2 2 2 2 2 2 2 2
6392 023/300 Read block 1381 len 32768
6393 024/300 Read block 1708 len 28672
6394 025/300 Read block 2101 len 36864
6395 026/300 Write block 891 len 24576 data: 3 3 3 4 4 4
6396 027/300 Write block 2157 len 8192 data: 3 3
6397 028/300 Flush
6398 029/300 Read block 4219 len 4096
6399 030/300 Flush
6400 031/300 Write block 3262 len 24576 data: 3 2 2 3 3 4
6401 032/300 Read block 2502 len 32768
6402 033/300 Read block 4674 len 20480
6403 034/300 Write block 1525 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6404 035/300 Flush
6405 036/300 Read block 3371 len 4096
6406 037/300 Read block 3896 len 12288
6407 038/300 Write block 72 len 40960 data: 3 3 3 3 3 3 3 3 3 3
6408 039/300 Write block 3081 len 16384 data: 3 3 3 3
6409 040/300 Read block 1058 len 8192
6410 041/300 Write block 3615 len 32768 data: 2 3 3 5 5 5 5 5
6411 042/300 Flush
6412 043/300 Read block 4897 len 12288
6413 044/300 Write block 1059 len 20480 data: 2 2 2 2 2
6414 045/300 Read block 296 len 12288
6415 046/300 Read block 3669 len 16384
6416 047/300 Read block 3806 len 16384
6417 048/300 Write block 2180 len 32768 data: 2 3 3 3 4 4 4 3
6418 049/300 Write block 740 len 24576 data: 2 2 2 2 2 3
6419 050/300 Write block 2485 len 32768 data: 2 2 2 2 2 2 2 2
6420 051/300 Write block 2104 len 4096 data: 2
6421 052/300 Write block 3828 len 4096 data: 3
6422 053/300 Read block 2600 len 28672
6423 054/300 Write block 122 len 36864 data: 3 3 3 2 2 2 2 2 2
6424 055/300 Read block 971 len 24576
6425 056/300 Write block 4258 len 36864 data: 3 2 2 2 2 2 2 2 2
6426 057/300 Write block 3440 len 16384 data: 3 3 3 3
6427 058/300 Read block 1634 len 40960
6428 059/300 Flush
6429 060/300 Write block 4198 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6430 061/300 Read block 120 len 24576
6431 062/300 Read block 2293 len 16384
6432 063/300 Write block 1356 len 24576 data: 5 5 5 6 5 3
6433 064/300 Write block 2193 len 20480 data: 2 2 2 2 2
6434 065/300 Write block 4802 len 24576 data: 3 3 2 2 2 2
6435 066/300 Read block 1927 len 40960
6436 067/300 Read block 1956 len 28672
6437 068/300 Read block 3365 len 20480
6438 069/300 Write block 1884 len 32768 data: 2 2 2 2 2 2 2 2
6439 070/300 Read block 453 len 20480
6440 071/300 Write block 2690 len 20480 data: 2 2 2 2 2
6441 072/300 Write block 1089 len 8192 data: 4 3
6442 073/300 Flush
6443 074/300 Read block 1773 len 4096
6444 075/300 Read block 4957 len 16384
6445 076/300 Read block 4266 len 20480
6446 077/300 Write block 3558 len 28672 data: 3 3 2 2 2 2 2
6447 078/300 Read block 131 len 24576
6448 079/300 Read block 4630 len 12288
6449 080/300 Write block 1631 len 40960 data: 3 2 3 3 3 3 3 3 3 3
6450 081/300 Write block 1235 len 36864 data: 3 2 2 2 2 2 2 2 2
6451 082/300 Read block 1652 len 40960
6452 083/300 Read block 2360 len 28672
6453 084/300 Write block 88 len 24576 data: 3 3 2 2 2 2
6454 085/300 Read block 611 len 12288
6455 086/300 Read block 2414 len 36864
6456 087/300 Flush
6457 088/300 Write block 4173 len 28672 data: 2 2 2 2 2 2 2
6458 089/300 Read block 3604 len 12288
6459 090/300 Read block 2783 len 16384
6460 091/300 Read block 1309 len 12288
6461 092/300 Read block 1370 len 4096
6462 093/300 Read block 4811 len 4096
6463 094/300 Read block 365 len 40960
6464 095/300 Read block 339 len 8192
6465 096/300 Write block 4488 len 4096 data: 2
6466 097/300 Flush
6467 098/300 Write block 25 len 40960 data: 4 4 4 4 3 3 3 3 3 2
6468 099/300 Read block 358 len 4096
6469 100/300 Write block 2590 len 4096 data: 2
6470 101/300 Write block 4604 len 8192 data: 5 4
6471 102/300 Write block 4526 len 28672 data: 4 4 4 2 2 2 2
6472 103/300 Read block 3835 len 28672
6473 104/300 Read block 547 len 12288
6474 105/300 Read block 3397 len 8192
6475 106/300 Flush
6476 107/300 Flush
6477 108/300 Write block 1769 len 32768 data: 3 3 3 2 2 2 2 2
6478 109/300 Read block 494 len 40960
6479 110/300 Read block 4841 len 36864
6480 111/300 Write block 2901 len 20480 data: 2 2 2 2 3
6481 112/300 Read block 78 len 20480
6482 113/300 Flush
6483 114/300 Flush
6484 115/300 Read block 351 len 12288
6485 116/300 Read block 4889 len 36864
6486 117/300 Read block 4323 len 8192
6487 118/300 Read block 1311 len 16384
6488 119/300 Read block 1590 len 4096
6489 120/300 Read block 4183 len 28672
6490 121/300 Read block 3969 len 24576
6491 122/300 Flush
6492 123/300 Read block 3850 len 4096
6493 124/300 Flush
6494 125/300 Read block 3003 len 40960
6495 126/300 Write block 454 len 16384 data: 4 3 4 4
6496 127/300 Read block 3405 len 40960
6497 128/300 Read block 1465 len 24576
6498 129/300 Write block 2598 len 28672 data: 2 2 2 2 2 3 2
6499 130/300 Read block 940 len 4096
6500 131/300 Read block 4923 len 12288
6501 132/300 Read block 1978 len 40960
6502 133/300 Write block 2752 len 28672 data: 2 2 2 2 2 2 2
6503 134/300 Read block 2940 len 28672
6504 135/300 Read block 1118 len 12288
6505 136/300 Read block 3879 len 32768
6506 137/300 Read block 1357 len 40960
6507 138/300 Read block 2429 len 40960
6508 139/300 Write block 3132 len 20480 data: 4 4 4 5 5
6509 140/300 Read block 1239 len 40960
6510 141/300 Write block 3331 len 32768 data: 3 3 3 3 4 3 2 2
6511 142/300 Read block 4775 len 36864
6512 143/300 Read block 2651 len 8192
6513 144/300 Read block 3201 len 32768
6514 145/300 Flush
6515 146/300 Read block 1146 len 40960
6516 147/300 Read block 1506 len 40960
6517 148/300 Write block 2301 len 24576 data: 3 3 3 4 4 4
6518 149/300 Read block 349 len 8192
6519 150/300 Write block 265 len 8192 data: 2 2
6520 151/300 Write block 4574 len 4096 data: 3
6521 152/300 Read block 539 len 20480
6522 153/300 Flush
6523 154/300 Read block 3266 len 8192
6524 155/300 Write block 1360 len 28672 data: 6 4 3 3 3 2 2
6525 156/300 Write block 1138 len 28672 data: 3 3 4 4 5 4 4
6526 157/300 Read block 563 len 40960
6527 158/300 Write block 1564 len 28672 data: 3 2 2 2 2 2 2
6528 159/300 Write block 2334 len 28672 data: 2 2 2 2 2 2 3
6529 160/300 Write block 949 len 36864 data: 3 3 4 4 4 4 4 4 3
6530 161/300 Read block 1251 len 16384
6531 162/300 Write block 237 len 16384 data: 3 3 2 2
6532 163/300 Write block 2957 len 16384 data: 2 2 2 2
6533 164/300 Write block 2189 len 4096 data: 2
6534 165/300 Read block 1633 len 32768
6535 166/300 Read block 1751 len 40960
6536 167/300 Flush
6537 168/300 Read block 4675 len 4096
6538 169/300 Read block 2564 len 12288
6539 170/300 Flush
6540 171/300 Flush
6541 172/300 Read block 1754 len 40960
6542 173/300 Write block 3506 len 36864 data: 2 2 2 3 3 3 3 2 2
6543 174/300 Flush
6544 175/300 Write block 883 len 12288 data: 2 2 2
6545 176/300 Write block 3991 len 32768 data: 3 2 2 2 2 2 2 2
6546 177/300 Read block 4500 len 36864
6547 178/300 Write block 4668 len 8192 data: 2 2
6548 179/300 Read block 1807 len 28672
6549 180/300 Read block 2334 len 32768
6550 181/300 Read block 814 len 36864
6551 182/300 Read block 3017 len 8192
6552 183/300 Read block 1328 len 32768
6553 184/300 Read block 3487 len 12288
6554 185/300 Flush
6555 186/300 Read block 532 len 28672
6556 187/300 Write block 4926 len 20480 data: 3 3 2 2 2
6557 188/300 Write block 1539 len 24576 data: 2 2 3 3 3 3
6558 189/300 Read block 738 len 8192
6559 190/300 Flush
6560 191/300 Write block 4133 len 36864 data: 2 2 2 2 2 2 2 2 2
6561 192/300 Write block 898 len 36864 data: 3 3 2 2 2 2 2 2 2
6562 193/300 Read block 4063 len 36864
6563 194/300 Read block 312 len 32768
6564 195/300 Read block 3782 len 28672
6565 196/300 Write block 3822 len 40960 data: 3 3 2 2 2 2 4 3 3 3
6566 197/300 Write block 3698 len 8192 data: 2 2
6567 198/300 Read block 1536 len 16384
6568 199/300 Read block 4490 len 8192
6569 200/300 Write block 790 len 40960 data: 4 4 4 4 4 4 4 3 2 2
6570 201/300 Read block 2207 len 24576
6571 202/300 Read block 4424 len 4096
6572 203/300 Read block 2215 len 4096
6573 204/300 Write block 2591 len 36864 data: 2 2 2 2 2 2 2 3 3
6574 205/300 Read block 4842 len 28672
6575 206/300 Read block 979 len 24576
6576 207/300 Write block 794 len 8192 data: 5 5
6577 208/300 Write block 3981 len 12288 data: 2 2 2
6578 209/300 Flush
6579 210/300 Flush
6580 211/300 Write block 3787 len 36864 data: 2 2 2 2 2 3 3 3 3
6581 212/300 Read block 3907 len 24576
6582 213/300 Read block 824 len 28672
6583 214/300 Write block 645 len 28672 data: 4 4 4 4 3 3 3
6584 215/300 Read block 3255 len 40960
6585 216/300 Read block 128 len 32768
6586 217/300 Write block 4464 len 24576 data: 2 2 2 2 2 2
6587 218/300 Read block 3542 len 32768
6588 219/300 Read block 2734 len 16384
6589 220/300 Read block 1329 len 28672
6590 221/300 Write block 1988 len 4096 data: 2
6591 222/300 Read block 2474 len 32768
6592 223/300 Read block 2642 len 32768
6593 224/300 Write block 3626 len 4096 data: 4
6594 225/300 Read block 2839 len 36864
6595 226/300 Read block 508 len 12288
6596 227/300 Write block 3053 len 16384 data: 3 3 3 4
6597 228/300 Read block 292 len 36864
6598 229/300 Read block 1532 len 24576
6599 230/300 Write block 899 len 12288 data: 4 3 3
6600 231/300 Write block 3784 len 12288 data: 2 2 2
6601 232/300 Read block 3799 len 28672
6602 233/300 Read block 3823 len 40960
6603 234/300 Write block 3236 len 28672 data: 4 3 3 2 3 3 3
6604 235/300 Read block 2015 len 4096
6605 236/300 Write block 512 len 20480 data: 2 2 2 3 3
6606 237/300 Write block 1448 len 24576 data: 5 5 5 2 2 2
6607 238/300 Write block 648 len 24576 data: 5 4 4 4 3 3
6608 239/300 Read block 3850 len 20480
6609 240/300 Read block 394 len 32768
6610 241/300 Write block 3879 len 24576 data: 3 3 3 4 4 3
6611 242/300 Read block 2804 len 36864
6612 243/300 Read block 4738 len 36864
6613 244/300 Read block 3000 len 12288
6614 245/300 Write block 1232 len 12288 data: 3 3 3
6615 246/300 Flush
6616 247/300 Flush
6617 248/300 Write block 4274 len 32768 data: 2 2 2 2 2 2 2 2
6618 249/300 Flush
6619 250/300 Flush
6620 251/300 Read block 4951 len 4096
6621 252/300 Read block 3287 len 12288
6622 253/300 Read block 2705 len 16384
6623 254/300 Flush
6624 255/300 Write block 3795 len 24576 data: 4 3 3 3 3 3
6625 256/300 Read block 3931 len 24576
6626 257/300 Read block 2689 len 4096
6627 258/300 Write block 480 len 36864 data: 3 3 4 3 3 4 4 4 4
6628 259/300 Write block 4431 len 24576 data: 2 3 3 3 4 4
6629 260/300 Write block 4384 len 40960 data: 3 3 2 2 2 2 2 2 2 2
6630 261/300 Read block 2025 len 20480
6631 262/300 Read block 4215 len 8192
6632 263/300 Write block 951 len 16384 data: 5 5 5 5
6633 264/300 Write block 4750 len 28672 data: 2 2 2 3 4 5 4
6634 265/300 Flush
6635 266/300 Read block 2625 len 8192
6636 267/300 Read block 3796 len 24576
6637 268/300 Read block 1026 len 12288
6638 269/300 Read block 869 len 36864
6639 270/300 Write block 1853 len 8192 data: 4 4
6640 271/300 Read block 1343 len 20480
6641 272/300 Read block 486 len 36864
6642 273/300 Write block 2742 len 36864 data: 3 3 3 3 3 3 3 2 2
6643 274/300 Write block 4250 len 12288 data: 3 4 4
6644 275/300 Write block 2028 len 20480 data: 3 3 2 2 3
6645 276/300 Write block 240 len 32768 data: 3 2 2 2 2 2 2 2
6646 277/300 Read block 4452 len 28672
6647 278/300 Read block 1527 len 20480
6648 279/300 Write block 3342 len 20480 data: 3 3 3 3 2
6649 280/300 Write block 2305 len 4096 data: 5
6650 281/300 Flush
6651 282/300 Write block 4559 len 16384 data: 5 4 2 2
6652 283/300 Flush
6653 284/300 Write block 3376 len 24576 data: 3 3 3 3 2 2
6654 285/300 Write block 2641 len 12288 data: 2 2 2
6655 286/300 Read block 2752 len 12288
6656 287/300 Read block 2376 len 40960
6657 288/300 Read block 34 len 20480
6658 289/300 Read block 184 len 8192
6659 290/300 Write block 432 len 36864 data: 4 3 2 2 3 3 5 5 5
6660 291/300 Flush
6661 292/300 Read block 1774 len 12288
6662 293/300 Read block 3698 len 32768
6663 294/300 Write block 1883 len 8192 data: 2 3
6664 295/300 Write block 3561 len 40960 data: 3 3 3 3 2 2 2 2 2 2
6665 296/300 Write block 1100 len 40960 data: 2 2 2 2 3 3 3 3 3 3
6666 297/300 Write block 3472 len 36864 data: 3 3 3 2 2 3 3 3 3
6667 298/300 Read block 3360 len 16384
6668 299/300 Read block 1389 len 40960
6669 300/300 Write block 4553 len 28672 data: 2 3 3 4 4 5 6
6670 301/300 Read block 2515 len 32768
6671 Main task has work to do, go find it
6672 got dsc Start(1)
6673 start 1
6674 Nov 07 14:04:43.571 INFO request completed, latency_us: 208, response_code: 204, uri: /start/cid/1, method: POST, req_id: d5479ab4-3a06-4c03-829e-bcb53cacad73, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
6675 Replay: started 1, returned:Ok(())
6676 ----------------------------------------------------------------
6677 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6678 States: Active Offline Active
6679 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6680 2512 Acked Write 7 Done Sent Done false
6681 2513 Acked Read 2 Done Sent Done false
6682 2514 Acked Read 3 Done Sent Done false
6683 2515 Acked Read 1 Done Sent Done false
6684 2516 Acked Flush 0 Done Sent Done false
6685 2517 Acked Write 6 Done Sent Done false
6686 2518 Acked Write 7 Done Sent Done false
6687 2519 Acked Read 9 Done Sent Done false
6688 2520 Acked Read 8 Done Sent Done false
6689 2521 Acked Write 8 Done Sent Done false
6690 2522 Acked Write 3 Done Sent Done false
6691 2523 Acked Write 2 Done Sent Done false
6692 2524 Acked Write 1 Done Sent Done false
6693 2525 Acked Write 8 Done Sent Done false
6694 2526 Acked Write 1 Done Sent Done false
6695 2527 Acked Read 1 Done Sent Done false
6696 2528 Acked Flush 0 Done Sent Done false
6697 2529 Acked Read 8 Done Sent Done false
6698 2530 Acked Flush 0 Done Sent Done false
6699 2531 Acked Read 8 Done Sent Done false
6700 2532 Acked Flush 0 Done Sent Done false
6701 2533 Acked Write 9 Done Sent Done false
6702 2534 Acked Read 8 Done Sent Done false
6703 2535 Acked Read 7 Done Sent Done false
6704 2536 Acked Read 9 Done Sent Done false
6705 2537 Acked Write 6 Done Sent Done false
6706 2538 Acked Write 2 Done Sent Done false
6707 2539 Acked Flush 0 Done Sent Done false
6708 2540 Acked Read 1 Done Sent Done false
6709 2541 Acked Flush 0 Done Sent Done false
6710 2542 Acked Write 6 Done Sent Done false
6711 2543 Acked Read 8 Done Sent Done false
6712 2544 Acked Read 5 Done Sent Done false
6713 2545 Acked Write 10 Done Sent Done false
6714 2546 Acked Flush 0 Done Sent Done false
6715 2547 Acked Read 1 Done Sent Done false
6716 2548 Acked Read 3 Done Sent Done false
6717 2549 Acked Write 10 Done Sent Done false
6718 2550 Acked Write 4 Done Sent Done false
6719 2551 Acked Read 2 Done Sent Done false
6720 2552 Acked Write 8 Done Sent Done false
6721 2553 Acked Flush 0 Done Sent Done false
6722 2554 Acked Read 3 Done Sent Done false
6723 2555 Acked Write 5 Done Sent Done false
6724 2556 Acked Read 3 Done Sent Done false
6725 2557 Acked Read 4 Done Sent Done false
6726 2558 Acked Read 4 Done Sent Done false
6727 2559 Acked Write 8 Done Sent Done false
6728 2560 Acked Write 6 Done Sent Done false
6729 2561 Acked Write 8 Done Sent Done false
6730 2562 Acked Write 1 Done Sent Done false
6731 2563 Acked Write 1 Done Sent Done false
6732 2564 Acked Read 7 Done Sent Done false
6733 2565 Acked Write 9 Done Sent Done false
6734 2566 Acked Read 6 Done Sent Done false
6735 2567 Acked Write 9 Done Sent Done false
6736 2568 Acked Write 4 Done Sent Done false
6737 2569 Acked Read 10 Done Sent Done false
6738 2570 Acked Flush 0 Done Sent Done false
6739 2571 Acked Write 10 Done Sent Done false
6740 2572 Acked Read 6 Done Sent Done false
6741 2573 Acked Read 4 Done Sent Done false
6742 2574 Acked Write 6 Done Sent Done false
6743 2575 Acked Write 5 Done Sent Done false
6744 2576 Acked Write 6 Done Sent Done false
6745 2577 Acked Read 10 Done Sent Done false
6746 2578 Acked Read 7 Done Sent Done false
6747 2579 Acked Read 5 Done Sent Done false
6748 2580 Acked Write 8 Done Sent Done false
6749 2581 Acked Read 5 Done Sent Done false
6750 2582 Acked Write 5 Done Sent Done false
6751 2583 Acked Write 2 Done Sent Done false
6752 2584 Acked Flush 0 Done Sent Done false
6753 2585 Acked Read 1 Done Sent Done false
6754 2586 Acked Read 4 Done Sent Done false
6755 2587 Acked Read 5 Done Sent Done false
6756 2588 Acked Write 7 Done Sent Done false
6757 2589 Acked Read 6 Done Sent Done false
6758 2590 Acked Read 3 Done Sent Done false
6759 2591 Acked Write 10 Done Sent Done false
6760 2592 Acked Write 9 Done Sent Done false
6761 2593 Acked Read 10 Done Sent Done false
6762 2594 Acked Read 7 Done Sent Done false
6763 2595 Acked Write 6 Done Sent Done false
6764 2596 Acked Read 3 Done Sent Done false
6765 2597 Acked Read 9 Done Sent Done false
6766 2598 Acked Flush 0 Done Sent Done false
6767 2599 Acked Write 7 Done Sent Done false
6768 2600 Acked Read 3 Done Sent Done false
6769 2601 Acked Read 4 Done Sent Done false
6770 2602 Acked Read 3 Done Sent Done false
6771 2603 Acked Read 1 Done Sent Done false
6772 2604 Acked Read 1 Done Sent Done false
6773 2605 Acked Read 10 Done Sent Done false
6774 2606 Acked Read 2 Done Sent Done false
6775 2607 Acked Write 1 Done Sent Done false
6776 2608 Acked Flush 0 Done Sent Done false
6777 2609 Acked Write 10 Done Sent Done false
6778 2610 Acked Read 1 Done Sent Done false
6779 2611 Acked Write 1 Done Sent Done false
6780 2612 Acked Write 2 Done Sent Done false
6781 2613 Acked Write 7 Done Sent Done false
6782 2614 Acked Read 7 Done Sent Done false
6783 2615 Acked Read 3 Done Sent Done false
6784 2616 Acked Read 2 Done Sent Done false
6785 2617 Acked Flush 0 Done Sent Done false
6786 2618 Acked Flush 0 Done Sent Done false
6787 2619 Acked Write 8 Done Sent Done false
6788 2620 Acked Read 10 Done Sent Done false
6789 2621 Acked Read 9 Done Sent Done false
6790 2622 Acked Write 5 Done Sent Done false
6791 2623 Acked Read 5 Done Sent Done false
6792 2624 Acked Flush 0 Done Sent Done false
6793 2625 Acked Flush 0 Done Sent Done false
6794 2626 Acked Read 3 Done Sent Done false
6795 2627 Acked Read 9 Done Sent Done false
6796 2628 Acked Read 2 Done Sent Done false
6797 2629 Acked Read 4 Done Sent Done false
6798 2630 Acked Read 1 Done Sent Done false
6799 2631 Acked Read 7 Done Sent Done false
6800 2632 Acked Read 6 Done Sent Done false
6801 2633 Acked Flush 0 Done Sent Done false
6802 2634 Acked Read 1 Done Sent Done false
6803 2635 Acked Flush 0 Done Sent Done false
6804 2636 Acked Read 10 Done Sent Done false
6805 2637 Acked Write 4 Done Sent Done false
6806 2638 Acked Read 10 Done Sent Done false
6807 2639 Acked Read 6 Done Sent Done false
6808 2640 Acked Flush 0 Done Sent Done false
6809 2641 Acked Write 7 Done Sent Done false
6810 2642 Acked Read 1 Done Sent Done false
6811 2643 Acked Read 3 Done Sent Done false
6812 2644 Acked Read 10 Done Sent Done false
6813 2645 Acked Write 7 Done Sent Done false
6814 2646 Acked Read 7 Done Sent Done false
6815 2647 Acked Read 3 Done Sent Done false
6816 2648 Acked Read 8 Done Sent Done false
6817 2649 Acked Read 10 Done Sent Done false
6818 2650 Acked Read 10 Done Sent Done false
6819 2651 Acked Write 5 Done Sent Done false
6820 2652 Acked Read 10 Done Sent Done false
6821 2653 Acked Write 8 Done Sent Done false
6822 2654 Acked Read 9 Done Sent Done false
6823 2655 Acked Read 2 Done Sent Done false
6824 2656 Acked Read 8 Done Sent Done false
6825 2657 Acked Flush 0 Done Sent Done false
6826 2658 Acked Read 10 Done Sent Done false
6827 2659 Acked Read 10 Done Sent Done false
6828 2660 Acked Write 6 Done Sent Done false
6829 2661 Acked Read 2 Done Sent Done false
6830 2662 Acked Write 2 Done Sent Done false
6831 2663 Acked Write 1 Done Sent Done false
6832 2664 Acked Read 5 Done Sent Done false
6833 2665 Acked Flush 0 Done Sent Done false
6834 2666 Acked Read 2 Done Sent Done false
6835 2667 Acked Write 7 Done Sent Done false
6836 2668 Acked Write 7 Done Sent Done false
6837 2669 Acked Read 10 Done Sent Done false
6838 2670 Acked Write 7 Done Sent Done false
6839 2671 Acked Write 7 Done Sent Done false
6840 2672 Acked Write 9 Done Sent Done false
6841 2673 Acked Read 4 Done Sent Done false
6842 2674 Acked Write 4 Done Sent Done false
6843 2675 Acked Write 4 Done Sent Done false
6844 2676 Acked Write 1 Done Sent Done false
6845 2677 Acked Read 8 Done Sent Done false
6846 2678 Acked Read 10 Done Sent Done false
6847 2679 Acked Flush 0 Done Sent Done false
6848 2680 Acked Read 1 Done Sent Done false
6849 2681 Acked Read 3 Done Sent Done false
6850 2682 Acked Flush 0 Done Sent Done false
6851 2683 Acked Flush 0 Done Sent Done false
6852 2684 Acked Read 10 Done Sent Done false
6853 2685 Acked Write 9 Done Sent Done false
6854 2686 Acked Flush 0 Done Sent Done false
6855 2687 Acked Write 3 Done Sent Done false
6856 2688 Acked Write 8 Done Sent Done false
6857 2689 Acked Read 9 Done Sent Done false
6858 2690 Acked Write 2 Done Sent Done false
6859 2691 Acked Read 7 Done Sent Done false
6860 2692 Acked Read 8 Done Sent Done false
6861 2693 Acked Read 9 Done Sent Done false
6862 2694 Acked Read 2 Done Sent Done false
6863 2695 Acked Read 8 Done Sent Done false
6864 2696 Acked Read 3 Done Sent Done false
6865 2697 Acked Flush 0 Done Sent Done false
6866 2698 Acked Read 7 Done Sent Done false
6867 2699 Acked Write 5 Done Sent Done false
6868 2700 Acked Write 6 Done Sent Done false
6869 2701 Acked Read 2 Done Sent Done false
6870 2702 Acked Flush 0 Done Sent Done false
6871 2703 Acked Write 9 Done Sent Done false
6872 2704 Acked Write 9 Done Sent Done false
6873 2705 Acked Read 9 Done Sent Done false
6874 2706 Acked Read 8 Done Sent Done false
6875 2707 Acked Read 7 Done Sent Done false
6876 2708 Acked Write 10 Done Sent Done false
6877 2709 Acked Write 2 Done Sent Done false
6878 2710 Acked Read 4 Done Sent Done false
6879 2711 Acked Read 2 Done Sent Done false
6880 2712 Acked Write 10 Done Sent Done false
6881 2713 Acked Read 6 Done Sent Done false
6882 2714 Acked Read 1 Done Sent Done false
6883 2715 Acked Read 1 Done Sent Done false
6884 2716 Acked Write 9 Done Sent Done false
6885 2717 Acked Read 7 Done Sent Done false
6886 2718 Acked Read 6 Done Sent Done false
6887 2719 Acked Write 2 Done Sent Done false
6888 2720 Acked Write 3 Done Sent Done false
6889 2721 Acked Flush 0 Done Sent Done false
6890 2722 Acked Flush 0 Done Sent Done false
6891 2723 Acked Write 9 Done Sent Done false
6892 2724 Acked Read 6 Done Sent Done false
6893 2725 Acked Read 7 Done Sent Done false
6894 2726 Acked Write 7 Done Sent Done false
6895 2727 Acked Read 10 Done Sent Done false
6896 2728 Acked Read 8 Done Sent Done false
6897 2729 Acked Write 6 Done Sent Done false
6898 2730 Acked Read 8 Done Sent Done false
6899 2731 Acked Read 4 Done Sent Done false
6900 2732 Acked Read 7 Done Sent Done false
6901 2733 Acked Write 1 Done Sent Done false
6902 2734 Acked Read 8 Done Sent Done false
6903 2735 Acked Read 8 Done Sent Done false
6904 2736 Acked Write 1 Done Sent Done false
6905 2737 Acked Read 9 Done Sent Done false
6906 2738 Acked Read 3 Done Sent Done false
6907 2739 Acked Write 4 Done Sent Done false
6908 2740 Acked Read 9 Done Sent Done false
6909 2741 Acked Read 6 Done Sent Done false
6910 2742 Acked Write 3 Done Sent Done false
6911 2743 Acked Write 3 Done Sent Done false
6912 2744 Acked Read 7 Done Sent Done false
6913 2745 Acked Read 10 Done Sent Done false
6914 2746 Acked Write 7 Done Sent Done false
6915 2747 Acked Read 1 Done Sent Done false
6916 2748 Acked Write 5 Done Sent Done false
6917 2749 Acked Write 6 Done Sent Done false
6918 2750 Acked Write 6 Done Sent Done false
6919 2751 Acked Read 5 Done Sent Done false
6920 2752 Acked Read 8 Done Sent Done false
6921 2753 Acked Write 6 Done Sent Done false
6922 2754 Acked Read 9 Done Sent Done false
6923 2755 Acked Read 9 Done Sent Done false
6924 2756 Acked Read 3 Done Sent Done false
6925 2757 Acked Write 3 Done Sent Done false
6926 2758 Acked Flush 0 Done Sent Done false
6927 2759 Acked Flush 0 Done Sent Done false
6928 2760 Acked Write 8 Done Sent Done false
6929 2761 Acked Flush 0 Done Sent Done false
6930 2762 Acked Flush 0 Done Sent Done false
6931 2763 Acked Read 1 Done Sent Done false
6932 2764 Acked Read 3 Done Sent Done false
6933 2765 Acked Read 4 Done Sent Done false
6934 2766 Acked Flush 0 Done Sent Done false
6935 2767 Acked Write 6 Done Sent Done false
6936 2768 Acked Read 6 Done Sent Done false
6937 2769 Acked Read 1 Done Sent Done false
6938 2770 Acked Write 9 Done Sent Done false
6939 2771 Acked Write 6 Done Sent Done false
6940 2772 Acked Write 10 Done Sent Done false
6941 2773 Acked Read 5 Done Sent Done false
6942 2774 Acked Read 2 Done Sent Done false
6943 2775 Acked Write 4 Done Sent Done false
6944 2776 Acked Write 7 Done Sent Done false
6945 2777 Acked Flush 0 Done Sent Done false
6946 2778 Acked Read 2 Done Sent Done false
6947 2779 Acked Read 6 Done Sent Done false
6948 2780 Acked Read 3 Done Sent Done false
6949 2781 Acked Read 9 Done Sent Done false
6950 2782 Acked Write 2 Done Sent Done false
6951 2783 Acked Read 5 Done Sent Done false
6952 2784 Acked Read 9 Done Sent Done false
6953 2785 Acked Write 9 Done Sent Done false
6954 2786 Acked Write 3 Done Sent Done false
6955 2787 Acked Write 5 Done Sent Done false
6956 2788 Acked Write 8 Done Sent Done false
6957 2789 Acked Read 7 Done Sent Done false
6958 2790 Acked Read 5 Done Sent Done false
6959 2791 Acked Write 5 Done Sent Done false
6960 2792 Acked Write 1 Done Sent Done false
6961 2793 Acked Flush 0 Done Sent Done false
6962 2794 Acked Write 4 Done Sent Done false
6963 2795 Acked Flush 0 Done Sent Done false
6964 2796 Acked Write 6 Done Sent Done false
6965 2797 Acked Write 3 Done Sent Done false
6966 2798 Acked Read 3 Done Sent Done false
6967 2799 Acked Read 10 Done Sent Done false
6968 2800 Acked Read 5 Done Sent Done false
6969 2801 Acked Read 2 Done Sent Done false
6970 2802 Acked Write 9 Done Sent Done false
6971 2803 Acked Flush 0 Done Sent Done false
6972 2804 Acked Read 3 Done Sent Done false
6973 2805 Acked Read 8 Done Sent Done false
6974 2806 Acked Write 2 Done Sent Done false
6975 2807 Acked Write 10 Done Sent Done false
6976 2808 Acked Write 10 Done Sent Done false
6977 2809 Acked Write 9 Done Sent Done false
6978 2810 Acked Read 4 Done Sent Done false
6979 2811 Acked Read 10 Done Sent Done false
6980 2812 Acked Write 7 Done Sent Done false
6981 2813 Acked Read 8 Done Sent Done false
6982 STATES DS:0 DS:1 DS:2 TOTAL
6983 Sent 0 302 0 302
6984 Done 302 0 302 604
6985 Skipped 0 0 0 0
6986 Error 0 0 0 0
6987 Last Flush: 2803 2511 2803
6988 Downstairs last five completed: 2511 2510 2509 2508 2507
6989 Upstairs last five completed: 2813 2812 2811 2810 2809
6990 CLIENT: Up:0 ds:302 act:2
6991 ----------------------------------------------------------------
6992 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
6993 States: Active Offline Active
6994 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6995 2512 Acked Write 7 Done Sent Done false
6996 2513 Acked Read 2 Done Sent Done false
6997 2514 Acked Read 3 Done Sent Done false
6998 2515 Acked Read 1 Done Sent Done false
6999 2516 Acked Flush 0 Done Sent Done false
7000 2517 Acked Write 6 Done Sent Done false
7001 2518 Acked Write 7 Done Sent Done false
7002 2519 Acked Read 9 Done Sent Done false
7003 2520 Acked Read 8 Done Sent Done false
7004 2521 Acked Write 8 Done Sent Done false
7005 2522 Acked Write 3 Done Sent Done false
7006 2523 Acked Write 2 Done Sent Done false
7007 2524 Acked Write 1 Done Sent Done false
7008 2525 Acked Write 8 Done Sent Done false
7009 2526 Acked Write 1 Done Sent Done false
7010 2527 Acked Read 1 Done Sent Done false
7011 2528 Acked Flush 0 Done Sent Done false
7012 2529 Acked Read 8 Done Sent Done false
7013 2530 Acked Flush 0 Done Sent Done false
7014 2531 Acked Read 8 Done Sent Done false
7015 2532 Acked Flush 0 Done Sent Done false
7016 2533 Acked Write 9 Done Sent Done false
7017 2534 Acked Read 8 Done Sent Done false
7018 2535 Acked Read 7 Done Sent Done false
7019 2536 Acked Read 9 Done Sent Done false
7020 2537 Acked Write 6 Done Sent Done false
7021 2538 Acked Write 2 Done Sent Done false
7022 2539 Acked Flush 0 Done Sent Done false
7023 2540 Acked Read 1 Done Sent Done false
7024 2541 Acked Flush 0 Done Sent Done false
7025 2542 Acked Write 6 Done Sent Done false
7026 2543 Acked Read 8 Done Sent Done false
7027 2544 Acked Read 5 Done Sent Done false
7028 2545 Acked Write 10 Done Sent Done false
7029 2546 Acked Flush 0 Done Sent Done false
7030 2547 Acked Read 1 Done Sent Done false
7031 2548 Acked Read 3 Done Sent Done false
7032 2549 Acked Write 10 Done Sent Done false
7033 2550 Acked Write 4 Done Sent Done false
7034 2551 Acked Read 2 Done Sent Done false
7035 2552 Acked Write 8 Done Sent Done false
7036 2553 Acked Flush 0 Done Sent Done false
7037 2554 Acked Read 3 Done Sent Done false
7038 2555 Acked Write 5 Done Sent Done false
7039 2556 Acked Read 3 Done Sent Done false
7040 2557 Acked Read 4 Done Sent Done false
7041 2558 Acked Read 4 Done Sent Done false
7042 2559 Acked Write 8 Done Sent Done false
7043 2560 Acked Write 6 Done Sent Done false
7044 2561 Acked Write 8 Done Sent Done false
7045 2562 Acked Write 1 Done Sent Done false
7046 2563 Acked Write 1 Done Sent Done false
7047 2564 Acked Read 7 Done Sent Done false
7048 2565 Acked Write 9 Done Sent Done false
7049 2566 Acked Read 6 Done Sent Done false
7050 2567 Acked Write 9 Done Sent Done false
7051 2568 Acked Write 4 Done Sent Done false
7052 2569 Acked Read 10 Done Sent Done false
7053 2570 Acked Flush 0 Done Sent Done false
7054 2571 Acked Write 10 Done Sent Done false
7055 2572 Acked Read 6 Done Sent Done false
7056 2573 Acked Read 4 Done Sent Done false
7057 2574 Acked Write 6 Done Sent Done false
7058 2575 Acked Write 5 Done Sent Done false
7059 2576 Acked Write 6 Done Sent Done false
7060 2577 Acked Read 10 Done Sent Done false
7061 2578 Acked Read 7 Done Sent Done false
7062 2579 Acked Read 5 Done Sent Done false
7063 2580 Acked Write 8 Done Sent Done false
7064 2581 Acked Read 5 Done Sent Done false
7065 2582 Acked Write 5 Done Sent Done false
7066 2583 Acked Write 2 Done Sent Done false
7067 2584 Acked Flush 0 Done Sent Done false
7068 2585 Acked Read 1 Done Sent Done false
7069 2586 Acked Read 4 Done Sent Done false
7070 2587 Acked Read 5 Done Sent Done false
7071 2588 Acked Write 7 Done Sent Done false
7072 2589 Acked Read 6 Done Sent Done false
7073 2590 Acked Read 3 Done Sent Done false
7074 2591 Acked Write 10 Done Sent Done false
7075 2592 Acked Write 9 Done Sent Done false
7076 2593 Acked Read 10 Done Sent Done false
7077 2594 Acked Read 7 Done Sent Done false
7078 2595 Acked Write 6 Done Sent Done false
7079 2596 Acked Read 3 Done Sent Done false
7080 2597 Acked Read 9 Done Sent Done false
7081 2598 Acked Flush 0 Done Sent Done false
7082 2599 Acked Write 7 Done Sent Done false
7083 2600 Acked Read 3 Done Sent Done false
7084 2601 Acked Read 4 Done Sent Done false
7085 2602 Acked Read 3 Done Sent Done false
7086 2603 Acked Read 1 Done Sent Done false
7087 2604 Acked Read 1 Done Sent Done false
7088 2605 Acked Read 10 Done Sent Done false
7089 2606 Acked Read 2 Done Sent Done false
7090 2607 Acked Write 1 Done Sent Done false
7091 2608 Acked Flush 0 Done Sent Done false
7092 2609 Acked Write 10 Done Sent Done false
7093 2610 Acked Read 1 Done Sent Done false
7094 2611 Acked Write 1 Done Sent Done false
7095 2612 Acked Write 2 Done Sent Done false
7096 2613 Acked Write 7 Done Sent Done false
7097 2614 Acked Read 7 Done Sent Done false
7098 2615 Acked Read 3 Done Sent Done false
7099 2616 Acked Read 2 Done Sent Done false
7100 2617 Acked Flush 0 Done Sent Done false
7101 2618 Acked Flush 0 Done Sent Done false
7102 2619 Acked Write 8 Done Sent Done false
7103 2620 Acked Read 10 Done Sent Done false
7104 2621 Acked Read 9 Done Sent Done false
7105 2622 Acked Write 5 Done Sent Done false
7106 2623 Acked Read 5 Done Sent Done false
7107 2624 Acked Flush 0 Done Sent Done false
7108 2625 Acked Flush 0 Done Sent Done false
7109 2626 Acked Read 3 Done Sent Done false
7110 2627 Acked Read 9 Done Sent Done false
7111 2628 Acked Read 2 Done Sent Done false
7112 2629 Acked Read 4 Done Sent Done false
7113 2630 Acked Read 1 Done Sent Done false
7114 2631 Acked Read 7 Done Sent Done false
7115 2632 Acked Read 6 Done Sent Done false
7116 2633 Acked Flush 0 Done Sent Done false
7117 2634 Acked Read 1 Done Sent Done false
7118 2635 Acked Flush 0 Done Sent Done false
7119 2636 Acked Read 10 Done Sent Done false
7120 2637 Acked Write 4 Done Sent Done false
7121 2638 Acked Read 10 Done Sent Done false
7122 2639 Acked Read 6 Done Sent Done false
7123 2640 Acked Flush 0 Done Sent Done false
7124 2641 Acked Write 7 Done Sent Done false
7125 2642 Acked Read 1 Done Sent Done false
7126 2643 Acked Read 3 Done Sent Done false
7127 2644 Acked Read 10 Done Sent Done false
7128 2645 Acked Write 7 Done Sent Done false
7129 2646 Acked Read 7 Done Sent Done false
7130 2647 Acked Read 3 Done Sent Done false
7131 2648 Acked Read 8 Done Sent Done false
7132 2649 Acked Read 10 Done Sent Done false
7133 2650 Acked Read 10 Done Sent Done false
7134 2651 Acked Write 5 Done Sent Done false
7135 2652 Acked Read 10 Done Sent Done false
7136 2653 Acked Write 8 Done Sent Done false
7137 2654 Acked Read 9 Done Sent Done false
7138 2655 Acked Read 2 Done Sent Done false
7139 2656 Acked Read 8 Done Sent Done false
7140 2657 Acked Flush 0 Done Sent Done false
7141 2658 Acked Read 10 Done Sent Done false
7142 2659 Acked Read 10 Done Sent Done false
7143 2660 Acked Write 6 Done Sent Done false
7144 2661 Acked Read 2 Done Sent Done false
7145 2662 Acked Write 2 Done Sent Done false
7146 2663 Acked Write 1 Done Sent Done false
7147 2664 Acked Read 5 Done Sent Done false
7148 2665 Acked Flush 0 Done Sent Done false
7149 2666 Acked Read 2 Done Sent Done false
7150 2667 Acked Write 7 Done Sent Done false
7151 2668 Acked Write 7 Done Sent Done false
7152 2669 Acked Read 10 Done Sent Done false
7153 2670 Acked Write 7 Done Sent Done false
7154 2671 Acked Write 7 Done Sent Done false
7155 2672 Acked Write 9 Done Sent Done false
7156 2673 Acked Read 4 Done Sent Done false
7157 2674 Acked Write 4 Done Sent Done false
7158 2675 Acked Write 4 Done Sent Done false
7159 2676 Acked Write 1 Done Sent Done false
7160 2677 Acked Read 8 Done Sent Done false
7161 2678 Acked Read 10 Done Sent Done false
7162 2679 Acked Flush 0 Done Sent Done false
7163 2680 Acked Read 1 Done Sent Done false
7164 2681 Acked Read 3 Done Sent Done false
7165 2682 Acked Flush 0 Done Sent Done false
7166 2683 Acked Flush 0 Done Sent Done false
7167 2684 Acked Read 10 Done Sent Done false
7168 2685 Acked Write 9 Done Sent Done false
7169 2686 Acked Flush 0 Done Sent Done false
7170 2687 Acked Write 3 Done Sent Done false
7171 2688 Acked Write 8 Done Sent Done false
7172 2689 Acked Read 9 Done Sent Done false
7173 2690 Acked Write 2 Done Sent Done false
7174 2691 Acked Read 7 Done Sent Done false
7175 2692 Acked Read 8 Done Sent Done false
7176 2693 Acked Read 9 Done Sent Done false
7177 2694 Acked Read 2 Done Sent Done false
7178 2695 Acked Read 8 Done Sent Done false
7179 2696 Acked Read 3 Done Sent Done false
7180 2697 Acked Flush 0 Done Sent Done false
7181 2698 Acked Read 7 Done Sent Done false
7182 2699 Acked Write 5 Done Sent Done false
7183 2700 Acked Write 6 Done Sent Done false
7184 2701 Acked Read 2 Done Sent Done false
7185 2702 Acked Flush 0 Done Sent Done false
7186 2703 Acked Write 9 Done Sent Done false
7187 2704 Acked Write 9 Done Sent Done false
7188 2705 Acked Read 9 Done Sent Done false
7189 2706 Acked Read 8 Done Sent Done false
7190 2707 Acked Read 7 Done Sent Done false
7191 2708 Acked Write 10 Done Sent Done false
7192 2709 Acked Write 2 Done Sent Done false
7193 2710 Acked Read 4 Done Sent Done false
7194 2711 Acked Read 2 Done Sent Done false
7195 2712 Acked Write 10 Done Sent Done false
7196 2713 Acked Read 6 Done Sent Done false
7197 2714 Acked Read 1 Done Sent Done false
7198 2715 Acked Read 1 Done Sent Done false
7199 2716 Acked Write 9 Done Sent Done false
7200 2717 Acked Read 7 Done Sent Done false
7201 2718 Acked Read 6 Done Sent Done false
7202 2719 Acked Write 2 Done Sent Done false
7203 2720 Acked Write 3 Done Sent Done false
7204 2721 Acked Flush 0 Done Sent Done false
7205 2722 Acked Flush 0 Done Sent Done false
7206 2723 Acked Write 9 Done Sent Done false
7207 2724 Acked Read 6 Done Sent Done false
7208 2725 Acked Read 7 Done Sent Done false
7209 2726 Acked Write 7 Done Sent Done false
7210 2727 Acked Read 10 Done Sent Done false
7211 2728 Acked Read 8 Done Sent Done false
7212 2729 Acked Write 6 Done Sent Done false
7213 2730 Acked Read 8 Done Sent Done false
7214 2731 Acked Read 4 Done Sent Done false
7215 2732 Acked Read 7 Done Sent Done false
7216 2733 Acked Write 1 Done Sent Done false
7217 2734 Acked Read 8 Done Sent Done false
7218 2735 Acked Read 8 Done Sent Done false
7219 2736 Acked Write 1 Done Sent Done false
7220 2737 Acked Read 9 Done Sent Done false
7221 2738 Acked Read 3 Done Sent Done false
7222 2739 Acked Write 4 Done Sent Done false
7223 2740 Acked Read 9 Done Sent Done false
7224 2741 Acked Read 6 Done Sent Done false
7225 2742 Acked Write 3 Done Sent Done false
7226 2743 Acked Write 3 Done Sent Done false
7227 2744 Acked Read 7 Done Sent Done false
7228 2745 Acked Read 10 Done Sent Done false
7229 2746 Acked Write 7 Done Sent Done false
7230 2747 Acked Read 1 Done Sent Done false
7231 2748 Acked Write 5 Done Sent Done false
7232 2749 Acked Write 6 Done Sent Done false
7233 2750 Acked Write 6 Done Sent Done false
7234 2751 Acked Read 5 Done Sent Done false
7235 2752 Acked Read 8 Done Sent Done false
7236 2753 Acked Write 6 Done Sent Done false
7237 2754 Acked Read 9 Done Sent Done false
7238 2755 Acked Read 9 Done Sent Done false
7239 2756 Acked Read 3 Done Sent Done false
7240 2757 Acked Write 3 Done Sent Done false
7241 2758 Acked Flush 0 Done Sent Done false
7242 2759 Acked Flush 0 Done Sent Done false
7243 2760 Acked Write 8 Done Sent Done false
7244 2761 Acked Flush 0 Done Sent Done false
7245 2762 Acked Flush 0 Done Sent Done false
7246 2763 Acked Read 1 Done Sent Done false
7247 2764 Acked Read 3 Done Sent Done false
7248 2765 Acked Read 4 Done Sent Done false
7249 2766 Acked Flush 0 Done Sent Done false
7250 2767 Acked Write 6 Done Sent Done false
7251 2768 Acked Read 6 Done Sent Done false
7252 2769 Acked Read 1 Done Sent Done false
7253 2770 Acked Write 9 Done Sent Done false
7254 2771 Acked Write 6 Done Sent Done false
7255 2772 Acked Write 10 Done Sent Done false
7256 2773 Acked Read 5 Done Sent Done false
7257 2774 Acked Read 2 Done Sent Done false
7258 2775 Acked Write 4 Done Sent Done false
7259 2776 Acked Write 7 Done Sent Done false
7260 2777 Acked Flush 0 Done Sent Done false
7261 2778 Acked Read 2 Done Sent Done false
7262 2779 Acked Read 6 Done Sent Done false
7263 2780 Acked Read 3 Done Sent Done false
7264 2781 Acked Read 9 Done Sent Done false
7265 2782 Acked Write 2 Done Sent Done false
7266 2783 Acked Read 5 Done Sent Done false
7267 2784 Acked Read 9 Done Sent Done false
7268 2785 Acked Write 9 Done Sent Done false
7269 2786 Acked Write 3 Done Sent Done false
7270 2787 Acked Write 5 Done Sent Done false
7271 2788 Acked Write 8 Done Sent Done false
7272 2789 Acked Read 7 Done Sent Done false
7273 2790 Acked Read 5 Done Sent Done false
7274 2791 Acked Write 5 Done Sent Done false
7275 2792 Acked Write 1 Done Sent Done false
7276 2793 Acked Flush 0 Done Sent Done false
7277 2794 Acked Write 4 Done Sent Done false
7278 2795 Acked Flush 0 Done Sent Done false
7279 2796 Acked Write 6 Done Sent Done false
7280 2797 Acked Write 3 Done Sent Done false
7281 2798 Acked Read 3 Done Sent Done false
7282 2799 Acked Read 10 Done Sent Done false
7283 2800 Acked Read 5 Done Sent Done false
7284 2801 Acked Read 2 Done Sent Done false
7285 2802 Acked Write 9 Done Sent Done false
7286 2803 Acked Flush 0 Done Sent Done false
7287 2804 Acked Read 3 Done Sent Done false
7288 2805 Acked Read 8 Done Sent Done false
7289 2806 Acked Write 2 Done Sent Done false
7290 2807 Acked Write 10 Done Sent Done false
7291 2808 Acked Write 10 Done Sent Done false
7292 2809 Acked Write 9 Done Sent Done false
7293 2810 Acked Read 4 Done Sent Done false
7294 2811 Acked Read 10 Done Sent Done false
7295 2812 Acked Write 7 Done Sent Done false
7296 2813 Acked Read 8 Done Sent Done false
7297 2814 Acked Flush 0 Done Sent Done false
7298 STATES DS:0 DS:1 DS:2 TOTAL
7299 Sent 0 303 0 303
7300 Done 303 0 303 606
7301 Skipped 0 0 0 0
7302 Error 0 0 0 0
7303 Last Flush: 2814 2511 2814
7304 Downstairs last five completed: 2511 2510 2509 2508 2507
7305 Upstairs last five completed: 2814 2813 2812 2811 2810
7306 CLIENT: Up:0 ds:303 act:2
73072024-11-07T14:04:49.277ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
73082024-11-07T14:04:49.278ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
73092024-11-07T14:04:49.278ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
73102024-11-07T14:04:49.278ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
73112024-11-07T14:04:49.278ZINFOcrucible: client task is exiting = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
7312 [8820] got start action
7313 [8820] I am going to restart
7314 Starting downstairs at port 8820
7315 Make output file at "/tmp/dsc/downstairs-8820.txt"
7316 [8820][1] reports Starting
7317 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1205)
7318 [8820][1] reports Running
7319 ----------------------------------------------------------------
7320 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
7321 States: Active Offline Active
7322 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7323 2512 Acked Write 7 Done Sent Done false
7324 2513 Acked Read 2 Done Sent Done false
7325 2514 Acked Read 3 Done Sent Done false
7326 2515 Acked Read 1 Done Sent Done false
7327 2516 Acked Flush 0 Done Sent Done false
7328 2517 Acked Write 6 Done Sent Done false
7329 2518 Acked Write 7 Done Sent Done false
7330 2519 Acked Read 9 Done Sent Done false
7331 2520 Acked Read 8 Done Sent Done false
7332 2521 Acked Write 8 Done Sent Done false
7333 2522 Acked Write 3 Done Sent Done false
7334 2523 Acked Write 2 Done Sent Done false
7335 2524 Acked Write 1 Done Sent Done false
7336 2525 Acked Write 8 Done Sent Done false
7337 2526 Acked Write 1 Done Sent Done false
7338 2527 Acked Read 1 Done Sent Done false
7339 2528 Acked Flush 0 Done Sent Done false
7340 2529 Acked Read 8 Done Sent Done false
7341 2530 Acked Flush 0 Done Sent Done false
7342 2531 Acked Read 8 Done Sent Done false
7343 2532 Acked Flush 0 Done Sent Done false
7344 2533 Acked Write 9 Done Sent Done false
7345 2534 Acked Read 8 Done Sent Done false
7346 2535 Acked Read 7 Done Sent Done false
7347 2536 Acked Read 9 Done Sent Done false
7348 2537 Acked Write 6 Done Sent Done false
7349 2538 Acked Write 2 Done Sent Done false
7350 2539 Acked Flush 0 Done Sent Done false
7351 2540 Acked Read 1 Done Sent Done false
7352 2541 Acked Flush 0 Done Sent Done false
7353 2542 Acked Write 6 Done Sent Done false
7354 2543 Acked Read 8 Done Sent Done false
7355 2544 Acked Read 5 Done Sent Done false
7356 2545 Acked Write 10 Done Sent Done false
7357 2546 Acked Flush 0 Done Sent Done false
7358 2547 Acked Read 1 Done Sent Done false
7359 2548 Acked Read 3 Done Sent Done false
7360 2549 Acked Write 10 Done Sent Done false
7361 2550 Acked Write 4 Done Sent Done false
7362 2551 Acked Read 2 Done Sent Done false
7363 2552 Acked Write 8 Done Sent Done false
7364 2553 Acked Flush 0 Done Sent Done false
7365 2554 Acked Read 3 Done Sent Done false
7366 2555 Acked Write 5 Done Sent Done false
7367 2556 Acked Read 3 Done Sent Done false
7368 2557 Acked Read 4 Done Sent Done false
7369 2558 Acked Read 4 Done Sent Done false
7370 2559 Acked Write 8 Done Sent Done false
7371 2560 Acked Write 6 Done Sent Done false
7372 2561 Acked Write 8 Done Sent Done false
7373 2562 Acked Write 1 Done Sent Done false
7374 2563 Acked Write 1 Done Sent Done false
7375 2564 Acked Read 7 Done Sent Done false
7376 2565 Acked Write 9 Done Sent Done false
7377 2566 Acked Read 6 Done Sent Done false
7378 2567 Acked Write 9 Done Sent Done false
7379 2568 Acked Write 4 Done Sent Done false
7380 2569 Acked Read 10 Done Sent Done false
7381 2570 Acked Flush 0 Done Sent Done false
7382 2571 Acked Write 10 Done Sent Done false
7383 2572 Acked Read 6 Done Sent Done false
7384 2573 Acked Read 4 Done Sent Done false
7385 2574 Acked Write 6 Done Sent Done false
7386 2575 Acked Write 5 Done Sent Done false
7387 2576 Acked Write 6 Done Sent Done false
7388 2577 Acked Read 10 Done Sent Done false
7389 2578 Acked Read 7 Done Sent Done false
7390 2579 Acked Read 5 Done Sent Done false
7391 2580 Acked Write 8 Done Sent Done false
7392 2581 Acked Read 5 Done Sent Done false
7393 2582 Acked Write 5 Done Sent Done false
7394 2583 Acked Write 2 Done Sent Done false
7395 2584 Acked Flush 0 Done Sent Done false
7396 2585 Acked Read 1 Done Sent Done false
7397 2586 Acked Read 4 Done Sent Done false
7398 2587 Acked Read 5 Done Sent Done false
7399 2588 Acked Write 7 Done Sent Done false
7400 2589 Acked Read 6 Done Sent Done false
7401 2590 Acked Read 3 Done Sent Done false
7402 2591 Acked Write 10 Done Sent Done false
7403 2592 Acked Write 9 Done Sent Done false
7404 2593 Acked Read 10 Done Sent Done false
7405 2594 Acked Read 7 Done Sent Done false
7406 2595 Acked Write 6 Done Sent Done false
7407 2596 Acked Read 3 Done Sent Done false
7408 2597 Acked Read 9 Done Sent Done false
7409 2598 Acked Flush 0 Done Sent Done false
7410 2599 Acked Write 7 Done Sent Done false
7411 2600 Acked Read 3 Done Sent Done false
7412 2601 Acked Read 4 Done Sent Done false
7413 2602 Acked Read 3 Done Sent Done false
7414 2603 Acked Read 1 Done Sent Done false
7415 2604 Acked Read 1 Done Sent Done false
7416 2605 Acked Read 10 Done Sent Done false
7417 2606 Acked Read 2 Done Sent Done false
7418 2607 Acked Write 1 Done Sent Done false
7419 2608 Acked Flush 0 Done Sent Done false
7420 2609 Acked Write 10 Done Sent Done false
7421 2610 Acked Read 1 Done Sent Done false
7422 2611 Acked Write 1 Done Sent Done false
7423 2612 Acked Write 2 Done Sent Done false
7424 2613 Acked Write 7 Done Sent Done false
7425 2614 Acked Read 7 Done Sent Done false
7426 2615 Acked Read 3 Done Sent Done false
7427 2616 Acked Read 2 Done Sent Done false
7428 2617 Acked Flush 0 Done Sent Done false
7429 2618 Acked Flush 0 Done Sent Done false
7430 2619 Acked Write 8 Done Sent Done false
7431 2620 Acked Read 10 Done Sent Done false
7432 2621 Acked Read 9 Done Sent Done false
7433 2622 Acked Write 5 Done Sent Done false
7434 2623 Acked Read 5 Done Sent Done false
7435 2624 Acked Flush 0 Done Sent Done false
7436 2625 Acked Flush 0 Done Sent Done false
7437 2626 Acked Read 3 Done Sent Done false
7438 2627 Acked Read 9 Done Sent Done false
7439 2628 Acked Read 2 Done Sent Done false
7440 2629 Acked Read 4 Done Sent Done false
7441 2630 Acked Read 1 Done Sent Done false
7442 2631 Acked Read 7 Done Sent Done false
7443 2632 Acked Read 6 Done Sent Done false
7444 2633 Acked Flush 0 Done Sent Done false
7445 2634 Acked Read 1 Done Sent Done false
7446 2635 Acked Flush 0 Done Sent Done false
7447 2636 Acked Read 10 Done Sent Done false
7448 2637 Acked Write 4 Done Sent Done false
7449 2638 Acked Read 10 Done Sent Done false
7450 2639 Acked Read 6 Done Sent Done false
7451 2640 Acked Flush 0 Done Sent Done false
7452 2641 Acked Write 7 Done Sent Done false
7453 2642 Acked Read 1 Done Sent Done false
7454 2643 Acked Read 3 Done Sent Done false
7455 2644 Acked Read 10 Done Sent Done false
7456 2645 Acked Write 7 Done Sent Done false
7457 2646 Acked Read 7 Done Sent Done false
7458 2647 Acked Read 3 Done Sent Done false
7459 2648 Acked Read 8 Done Sent Done false
7460 2649 Acked Read 10 Done Sent Done false
7461 2650 Acked Read 10 Done Sent Done false
7462 2651 Acked Write 5 Done Sent Done false
7463 2652 Acked Read 10 Done Sent Done false
7464 2653 Acked Write 8 Done Sent Done false
7465 2654 Acked Read 9 Done Sent Done false
7466 2655 Acked Read 2 Done Sent Done false
7467 2656 Acked Read 8 Done Sent Done false
7468 2657 Acked Flush 0 Done Sent Done false
7469 2658 Acked Read 10 Done Sent Done false
7470 2659 Acked Read 10 Done Sent Done false
7471 2660 Acked Write 6 Done Sent Done false
7472 2661 Acked Read 2 Done Sent Done false
7473 2662 Acked Write 2 Done Sent Done false
7474 2663 Acked Write 1 Done Sent Done false
7475 2664 Acked Read 5 Done Sent Done false
7476 2665 Acked Flush 0 Done Sent Done false
7477 2666 Acked Read 2 Done Sent Done false
7478 2667 Acked Write 7 Done Sent Done false
7479 2668 Acked Write 7 Done Sent Done false
7480 2669 Acked Read 10 Done Sent Done false
7481 2670 Acked Write 7 Done Sent Done false
7482 2671 Acked Write 7 Done Sent Done false
7483 2672 Acked Write 9 Done Sent Done false
7484 2673 Acked Read 4 Done Sent Done false
7485 2674 Acked Write 4 Done Sent Done false
7486 2675 Acked Write 4 Done Sent Done false
7487 2676 Acked Write 1 Done Sent Done false
7488 2677 Acked Read 8 Done Sent Done false
7489 2678 Acked Read 10 Done Sent Done false
7490 2679 Acked Flush 0 Done Sent Done false
7491 2680 Acked Read 1 Done Sent Done false
7492 2681 Acked Read 3 Done Sent Done false
7493 2682 Acked Flush 0 Done Sent Done false
7494 2683 Acked Flush 0 Done Sent Done false
7495 2684 Acked Read 10 Done Sent Done false
7496 2685 Acked Write 9 Done Sent Done false
7497 2686 Acked Flush 0 Done Sent Done false
7498 2687 Acked Write 3 Done Sent Done false
7499 2688 Acked Write 8 Done Sent Done false
7500 2689 Acked Read 9 Done Sent Done false
7501 2690 Acked Write 2 Done Sent Done false
7502 2691 Acked Read 7 Done Sent Done false
7503 2692 Acked Read 8 Done Sent Done false
7504 2693 Acked Read 9 Done Sent Done false
7505 2694 Acked Read 2 Done Sent Done false
7506 2695 Acked Read 8 Done Sent Done false
7507 2696 Acked Read 3 Done Sent Done false
7508 2697 Acked Flush 0 Done Sent Done false
7509 2698 Acked Read 7 Done Sent Done false
7510 2699 Acked Write 5 Done Sent Done false
7511 2700 Acked Write 6 Done Sent Done false
7512 2701 Acked Read 2 Done Sent Done false
7513 2702 Acked Flush 0 Done Sent Done false
7514 2703 Acked Write 9 Done Sent Done false
7515 2704 Acked Write 9 Done Sent Done false
7516 2705 Acked Read 9 Done Sent Done false
7517 2706 Acked Read 8 Done Sent Done false
7518 2707 Acked Read 7 Done Sent Done false
7519 2708 Acked Write 10 Done Sent Done false
7520 2709 Acked Write 2 Done Sent Done false
7521 2710 Acked Read 4 Done Sent Done false
7522 2711 Acked Read 2 Done Sent Done false
7523 2712 Acked Write 10 Done Sent Done false
7524 2713 Acked Read 6 Done Sent Done false
7525 2714 Acked Read 1 Done Sent Done false
7526 2715 Acked Read 1 Done Sent Done false
7527 2716 Acked Write 9 Done Sent Done false
7528 2717 Acked Read 7 Done Sent Done false
7529 2718 Acked Read 6 Done Sent Done false
7530 2719 Acked Write 2 Done Sent Done false
7531 2720 Acked Write 3 Done Sent Done false
7532 2721 Acked Flush 0 Done Sent Done false
7533 2722 Acked Flush 0 Done Sent Done false
7534 2723 Acked Write 9 Done Sent Done false
7535 2724 Acked Read 6 Done Sent Done false
7536 2725 Acked Read 7 Done Sent Done false
7537 2726 Acked Write 7 Done Sent Done false
7538 2727 Acked Read 10 Done Sent Done false
7539 2728 Acked Read 8 Done Sent Done false
7540 2729 Acked Write 6 Done Sent Done false
7541 2730 Acked Read 8 Done Sent Done false
7542 2731 Acked Read 4 Done Sent Done false
7543 2732 Acked Read 7 Done Sent Done false
7544 2733 Acked Write 1 Done Sent Done false
7545 2734 Acked Read 8 Done Sent Done false
7546 2735 Acked Read 8 Done Sent Done false
7547 2736 Acked Write 1 Done Sent Done false
7548 2737 Acked Read 9 Done Sent Done false
7549 2738 Acked Read 3 Done Sent Done false
7550 2739 Acked Write 4 Done Sent Done false
7551 2740 Acked Read 9 Done Sent Done false
7552 2741 Acked Read 6 Done Sent Done false
7553 2742 Acked Write 3 Done Sent Done false
7554 2743 Acked Write 3 Done Sent Done false
7555 2744 Acked Read 7 Done Sent Done false
7556 2745 Acked Read 10 Done Sent Done false
7557 2746 Acked Write 7 Done Sent Done false
7558 2747 Acked Read 1 Done Sent Done false
7559 2748 Acked Write 5 Done Sent Done false
7560 2749 Acked Write 6 Done Sent Done false
7561 2750 Acked Write 6 Done Sent Done false
7562 2751 Acked Read 5 Done Sent Done false
7563 2752 Acked Read 8 Done Sent Done false
7564 2753 Acked Write 6 Done Sent Done false
7565 2754 Acked Read 9 Done Sent Done false
7566 2755 Acked Read 9 Done Sent Done false
7567 2756 Acked Read 3 Done Sent Done false
7568 2757 Acked Write 3 Done Sent Done false
7569 2758 Acked Flush 0 Done Sent Done false
7570 2759 Acked Flush 0 Done Sent Done false
7571 2760 Acked Write 8 Done Sent Done false
7572 2761 Acked Flush 0 Done Sent Done false
7573 2762 Acked Flush 0 Done Sent Done false
7574 2763 Acked Read 1 Done Sent Done false
7575 2764 Acked Read 3 Done Sent Done false
7576 2765 Acked Read 4 Done Sent Done false
7577 2766 Acked Flush 0 Done Sent Done false
7578 2767 Acked Write 6 Done Sent Done false
7579 2768 Acked Read 6 Done Sent Done false
7580 2769 Acked Read 1 Done Sent Done false
7581 2770 Acked Write 9 Done Sent Done false
7582 2771 Acked Write 6 Done Sent Done false
7583 2772 Acked Write 10 Done Sent Done false
7584 2773 Acked Read 5 Done Sent Done false
7585 2774 Acked Read 2 Done Sent Done false
7586 2775 Acked Write 4 Done Sent Done false
7587 2776 Acked Write 7 Done Sent Done false
7588 2777 Acked Flush 0 Done Sent Done false
7589 2778 Acked Read 2 Done Sent Done false
7590 2779 Acked Read 6 Done Sent Done false
7591 2780 Acked Read 3 Done Sent Done false
7592 2781 Acked Read 9 Done Sent Done false
7593 2782 Acked Write 2 Done Sent Done false
7594 2783 Acked Read 5 Done Sent Done false
7595 2784 Acked Read 9 Done Sent Done false
7596 2785 Acked Write 9 Done Sent Done false
7597 2786 Acked Write 3 Done Sent Done false
7598 2787 Acked Write 5 Done Sent Done false
7599 2788 Acked Write 8 Done Sent Done false
7600 2789 Acked Read 7 Done Sent Done false
7601 2790 Acked Read 5 Done Sent Done false
7602 2791 Acked Write 5 Done Sent Done false
7603 2792 Acked Write 1 Done Sent Done false
7604 2793 Acked Flush 0 Done Sent Done false
7605 2794 Acked Write 4 Done Sent Done false
7606 2795 Acked Flush 0 Done Sent Done false
7607 2796 Acked Write 6 Done Sent Done false
7608 2797 Acked Write 3 Done Sent Done false
7609 2798 Acked Read 3 Done Sent Done false
7610 2799 Acked Read 10 Done Sent Done false
7611 2800 Acked Read 5 Done Sent Done false
7612 2801 Acked Read 2 Done Sent Done false
7613 2802 Acked Write 9 Done Sent Done false
7614 2803 Acked Flush 0 Done Sent Done false
7615 2804 Acked Read 3 Done Sent Done false
7616 2805 Acked Read 8 Done Sent Done false
7617 2806 Acked Write 2 Done Sent Done false
7618 2807 Acked Write 10 Done Sent Done false
7619 2808 Acked Write 10 Done Sent Done false
7620 2809 Acked Write 9 Done Sent Done false
7621 2810 Acked Read 4 Done Sent Done false
7622 2811 Acked Read 10 Done Sent Done false
7623 2812 Acked Write 7 Done Sent Done false
7624 2813 Acked Read 8 Done Sent Done false
7625 2814 Acked Flush 0 Done Sent Done false
7626 STATES DS:0 DS:1 DS:2 TOTAL
7627 Sent 0 303 0 303
7628 Done 303 0 303 606
7629 Skipped 0 0 0 0
7630 Error 0 0 0 0
7631 Last Flush: 2814 2511 2814
7632 Downstairs last five completed: 2511 2510 2509 2508 2507
7633 Upstairs last five completed: 2814 2813 2812 2811 2810
7634 CLIENT: Up:0 ds:303 act:2
7635 ----------------------------------------------------------------
7636 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
7637 States: Active Offline Active
7638 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7639 2512 Acked Write 7 Done Sent Done false
7640 2513 Acked Read 2 Done Sent Done false
7641 2514 Acked Read 3 Done Sent Done false
7642 2515 Acked Read 1 Done Sent Done false
7643 2516 Acked Flush 0 Done Sent Done false
7644 2517 Acked Write 6 Done Sent Done false
7645 2518 Acked Write 7 Done Sent Done false
7646 2519 Acked Read 9 Done Sent Done false
7647 2520 Acked Read 8 Done Sent Done false
7648 2521 Acked Write 8 Done Sent Done false
7649 2522 Acked Write 3 Done Sent Done false
7650 2523 Acked Write 2 Done Sent Done false
7651 2524 Acked Write 1 Done Sent Done false
7652 2525 Acked Write 8 Done Sent Done false
7653 2526 Acked Write 1 Done Sent Done false
7654 2527 Acked Read 1 Done Sent Done false
7655 2528 Acked Flush 0 Done Sent Done false
7656 2529 Acked Read 8 Done Sent Done false
7657 2530 Acked Flush 0 Done Sent Done false
7658 2531 Acked Read 8 Done Sent Done false
7659 2532 Acked Flush 0 Done Sent Done false
7660 2533 Acked Write 9 Done Sent Done false
7661 2534 Acked Read 8 Done Sent Done false
7662 2535 Acked Read 7 Done Sent Done false
7663 2536 Acked Read 9 Done Sent Done false
7664 2537 Acked Write 6 Done Sent Done false
7665 2538 Acked Write 2 Done Sent Done false
7666 2539 Acked Flush 0 Done Sent Done false
7667 2540 Acked Read 1 Done Sent Done false
7668 2541 Acked Flush 0 Done Sent Done false
7669 2542 Acked Write 6 Done Sent Done false
7670 2543 Acked Read 8 Done Sent Done false
7671 2544 Acked Read 5 Done Sent Done false
7672 2545 Acked Write 10 Done Sent Done false
7673 2546 Acked Flush 0 Done Sent Done false
7674 2547 Acked Read 1 Done Sent Done false
7675 2548 Acked Read 3 Done Sent Done false
7676 2549 Acked Write 10 Done Sent Done false
7677 2550 Acked Write 4 Done Sent Done false
7678 2551 Acked Read 2 Done Sent Done false
7679 2552 Acked Write 8 Done Sent Done false
7680 2553 Acked Flush 0 Done Sent Done false
7681 2554 Acked Read 3 Done Sent Done false
7682 2555 Acked Write 5 Done Sent Done false
7683 2556 Acked Read 3 Done Sent Done false
7684 2557 Acked Read 4 Done Sent Done false
7685 2558 Acked Read 4 Done Sent Done false
7686 2559 Acked Write 8 Done Sent Done false
7687 2560 Acked Write 6 Done Sent Done false
7688 2561 Acked Write 8 Done Sent Done false
7689 2562 Acked Write 1 Done Sent Done false
7690 2563 Acked Write 1 Done Sent Done false
7691 2564 Acked Read 7 Done Sent Done false
7692 2565 Acked Write 9 Done Sent Done false
7693 2566 Acked Read 6 Done Sent Done false
7694 2567 Acked Write 9 Done Sent Done false
7695 2568 Acked Write 4 Done Sent Done false
7696 2569 Acked Read 10 Done Sent Done false
7697 2570 Acked Flush 0 Done Sent Done false
7698 2571 Acked Write 10 Done Sent Done false
7699 2572 Acked Read 6 Done Sent Done false
7700 2573 Acked Read 4 Done Sent Done false
7701 2574 Acked Write 6 Done Sent Done false
7702 2575 Acked Write 5 Done Sent Done false
7703 2576 Acked Write 6 Done Sent Done false
7704 2577 Acked Read 10 Done Sent Done false
7705 2578 Acked Read 7 Done Sent Done false
7706 2579 Acked Read 5 Done Sent Done false
7707 2580 Acked Write 8 Done Sent Done false
7708 2581 Acked Read 5 Done Sent Done false
7709 2582 Acked Write 5 Done Sent Done false
7710 2583 Acked Write 2 Done Sent Done false
7711 2584 Acked Flush 0 Done Sent Done false
7712 2585 Acked Read 1 Done Sent Done false
7713 2586 Acked Read 4 Done Sent Done false
7714 2587 Acked Read 5 Done Sent Done false
7715 2588 Acked Write 7 Done Sent Done false
7716 2589 Acked Read 6 Done Sent Done false
7717 2590 Acked Read 3 Done Sent Done false
7718 2591 Acked Write 10 Done Sent Done false
7719 2592 Acked Write 9 Done Sent Done false
7720 2593 Acked Read 10 Done Sent Done false
7721 2594 Acked Read 7 Done Sent Done false
7722 2595 Acked Write 6 Done Sent Done false
7723 2596 Acked Read 3 Done Sent Done false
7724 2597 Acked Read 9 Done Sent Done false
7725 2598 Acked Flush 0 Done Sent Done false
7726 2599 Acked Write 7 Done Sent Done false
7727 2600 Acked Read 3 Done Sent Done false
7728 2601 Acked Read 4 Done Sent Done false
7729 2602 Acked Read 3 Done Sent Done false
7730 2603 Acked Read 1 Done Sent Done false
7731 2604 Acked Read 1 Done Sent Done false
7732 2605 Acked Read 10 Done Sent Done false
7733 2606 Acked Read 2 Done Sent Done false
7734 2607 Acked Write 1 Done Sent Done false
7735 2608 Acked Flush 0 Done Sent Done false
7736 2609 Acked Write 10 Done Sent Done false
7737 2610 Acked Read 1 Done Sent Done false
7738 2611 Acked Write 1 Done Sent Done false
7739 2612 Acked Write 2 Done Sent Done false
7740 2613 Acked Write 7 Done Sent Done false
7741 2614 Acked Read 7 Done Sent Done false
7742 2615 Acked Read 3 Done Sent Done false
7743 2616 Acked Read 2 Done Sent Done false
7744 2617 Acked Flush 0 Done Sent Done false
7745 2618 Acked Flush 0 Done Sent Done false
7746 2619 Acked Write 8 Done Sent Done false
7747 2620 Acked Read 10 Done Sent Done false
7748 2621 Acked Read 9 Done Sent Done false
7749 2622 Acked Write 5 Done Sent Done false
7750 2623 Acked Read 5 Done Sent Done false
7751 2624 Acked Flush 0 Done Sent Done false
7752 2625 Acked Flush 0 Done Sent Done false
7753 2626 Acked Read 3 Done Sent Done false
7754 2627 Acked Read 9 Done Sent Done false
7755 2628 Acked Read 2 Done Sent Done false
7756 2629 Acked Read 4 Done Sent Done false
7757 2630 Acked Read 1 Done Sent Done false
7758 2631 Acked Read 7 Done Sent Done false
7759 2632 Acked Read 6 Done Sent Done false
7760 2633 Acked Flush 0 Done Sent Done false
7761 2634 Acked Read 1 Done Sent Done false
7762 2635 Acked Flush 0 Done Sent Done false
7763 2636 Acked Read 10 Done Sent Done false
7764 2637 Acked Write 4 Done Sent Done false
7765 2638 Acked Read 10 Done Sent Done false
7766 2639 Acked Read 6 Done Sent Done false
7767 2640 Acked Flush 0 Done Sent Done false
7768 2641 Acked Write 7 Done Sent Done false
7769 2642 Acked Read 1 Done Sent Done false
7770 2643 Acked Read 3 Done Sent Done false
7771 2644 Acked Read 10 Done Sent Done false
7772 2645 Acked Write 7 Done Sent Done false
7773 2646 Acked Read 7 Done Sent Done false
7774 2647 Acked Read 3 Done Sent Done false
7775 2648 Acked Read 8 Done Sent Done false
7776 2649 Acked Read 10 Done Sent Done false
7777 2650 Acked Read 10 Done Sent Done false
7778 2651 Acked Write 5 Done Sent Done false
7779 2652 Acked Read 10 Done Sent Done false
7780 2653 Acked Write 8 Done Sent Done false
7781 2654 Acked Read 9 Done Sent Done false
7782 2655 Acked Read 2 Done Sent Done false
7783 2656 Acked Read 8 Done Sent Done false
7784 2657 Acked Flush 0 Done Sent Done false
7785 2658 Acked Read 10 Done Sent Done false
7786 2659 Acked Read 10 Done Sent Done false
7787 2660 Acked Write 6 Done Sent Done false
7788 2661 Acked Read 2 Done Sent Done false
7789 2662 Acked Write 2 Done Sent Done false
7790 2663 Acked Write 1 Done Sent Done false
7791 2664 Acked Read 5 Done Sent Done false
7792 2665 Acked Flush 0 Done Sent Done false
7793 2666 Acked Read 2 Done Sent Done false
7794 2667 Acked Write 7 Done Sent Done false
7795 2668 Acked Write 7 Done Sent Done false
7796 2669 Acked Read 10 Done Sent Done false
7797 2670 Acked Write 7 Done Sent Done false
7798 2671 Acked Write 7 Done Sent Done false
7799 2672 Acked Write 9 Done Sent Done false
7800 2673 Acked Read 4 Done Sent Done false
7801 2674 Acked Write 4 Done Sent Done false
7802 2675 Acked Write 4 Done Sent Done false
7803 2676 Acked Write 1 Done Sent Done false
7804 2677 Acked Read 8 Done Sent Done false
7805 2678 Acked Read 10 Done Sent Done false
7806 2679 Acked Flush 0 Done Sent Done false
7807 2680 Acked Read 1 Done Sent Done false
7808 2681 Acked Read 3 Done Sent Done false
7809 2682 Acked Flush 0 Done Sent Done false
7810 2683 Acked Flush 0 Done Sent Done false
7811 2684 Acked Read 10 Done Sent Done false
7812 2685 Acked Write 9 Done Sent Done false
7813 2686 Acked Flush 0 Done Sent Done false
7814 2687 Acked Write 3 Done Sent Done false
7815 2688 Acked Write 8 Done Sent Done false
7816 2689 Acked Read 9 Done Sent Done false
7817 2690 Acked Write 2 Done Sent Done false
7818 2691 Acked Read 7 Done Sent Done false
7819 2692 Acked Read 8 Done Sent Done false
7820 2693 Acked Read 9 Done Sent Done false
7821 2694 Acked Read 2 Done Sent Done false
7822 2695 Acked Read 8 Done Sent Done false
7823 2696 Acked Read 3 Done Sent Done false
7824 2697 Acked Flush 0 Done Sent Done false
7825 2698 Acked Read 7 Done Sent Done false
7826 2699 Acked Write 5 Done Sent Done false
7827 2700 Acked Write 6 Done Sent Done false
7828 2701 Acked Read 2 Done Sent Done false
7829 2702 Acked Flush 0 Done Sent Done false
7830 2703 Acked Write 9 Done Sent Done false
7831 2704 Acked Write 9 Done Sent Done false
7832 2705 Acked Read 9 Done Sent Done false
7833 2706 Acked Read 8 Done Sent Done false
7834 2707 Acked Read 7 Done Sent Done false
7835 2708 Acked Write 10 Done Sent Done false
7836 2709 Acked Write 2 Done Sent Done false
7837 2710 Acked Read 4 Done Sent Done false
7838 2711 Acked Read 2 Done Sent Done false
7839 2712 Acked Write 10 Done Sent Done false
7840 2713 Acked Read 6 Done Sent Done false
7841 2714 Acked Read 1 Done Sent Done false
7842 2715 Acked Read 1 Done Sent Done false
7843 2716 Acked Write 9 Done Sent Done false
7844 2717 Acked Read 7 Done Sent Done false
7845 2718 Acked Read 6 Done Sent Done false
7846 2719 Acked Write 2 Done Sent Done false
7847 2720 Acked Write 3 Done Sent Done false
7848 2721 Acked Flush 0 Done Sent Done false
7849 2722 Acked Flush 0 Done Sent Done false
7850 2723 Acked Write 9 Done Sent Done false
7851 2724 Acked Read 6 Done Sent Done false
7852 2725 Acked Read 7 Done Sent Done false
7853 2726 Acked Write 7 Done Sent Done false
7854 2727 Acked Read 10 Done Sent Done false
7855 2728 Acked Read 8 Done Sent Done false
7856 2729 Acked Write 6 Done Sent Done false
7857 2730 Acked Read 8 Done Sent Done false
7858 2731 Acked Read 4 Done Sent Done false
7859 2732 Acked Read 7 Done Sent Done false
7860 2733 Acked Write 1 Done Sent Done false
7861 2734 Acked Read 8 Done Sent Done false
7862 2735 Acked Read 8 Done Sent Done false
7863 2736 Acked Write 1 Done Sent Done false
7864 2737 Acked Read 9 Done Sent Done false
7865 2738 Acked Read 3 Done Sent Done false
7866 2739 Acked Write 4 Done Sent Done false
7867 2740 Acked Read 9 Done Sent Done false
7868 2741 Acked Read 6 Done Sent Done false
7869 2742 Acked Write 3 Done Sent Done false
7870 2743 Acked Write 3 Done Sent Done false
7871 2744 Acked Read 7 Done Sent Done false
7872 2745 Acked Read 10 Done Sent Done false
7873 2746 Acked Write 7 Done Sent Done false
7874 2747 Acked Read 1 Done Sent Done false
7875 2748 Acked Write 5 Done Sent Done false
7876 2749 Acked Write 6 Done Sent Done false
7877 2750 Acked Write 6 Done Sent Done false
7878 2751 Acked Read 5 Done Sent Done false
7879 2752 Acked Read 8 Done Sent Done false
7880 2753 Acked Write 6 Done Sent Done false
7881 2754 Acked Read 9 Done Sent Done false
7882 2755 Acked Read 9 Done Sent Done false
7883 2756 Acked Read 3 Done Sent Done false
7884 2757 Acked Write 3 Done Sent Done false
7885 2758 Acked Flush 0 Done Sent Done false
7886 2759 Acked Flush 0 Done Sent Done false
7887 2760 Acked Write 8 Done Sent Done false
7888 2761 Acked Flush 0 Done Sent Done false
7889 2762 Acked Flush 0 Done Sent Done false
7890 2763 Acked Read 1 Done Sent Done false
7891 2764 Acked Read 3 Done Sent Done false
7892 2765 Acked Read 4 Done Sent Done false
7893 2766 Acked Flush 0 Done Sent Done false
7894 2767 Acked Write 6 Done Sent Done false
7895 2768 Acked Read 6 Done Sent Done false
7896 2769 Acked Read 1 Done Sent Done false
7897 2770 Acked Write 9 Done Sent Done false
7898 2771 Acked Write 6 Done Sent Done false
7899 2772 Acked Write 10 Done Sent Done false
7900 2773 Acked Read 5 Done Sent Done false
7901 2774 Acked Read 2 Done Sent Done false
7902 2775 Acked Write 4 Done Sent Done false
7903 2776 Acked Write 7 Done Sent Done false
7904 2777 Acked Flush 0 Done Sent Done false
7905 2778 Acked Read 2 Done Sent Done false
7906 2779 Acked Read 6 Done Sent Done false
7907 2780 Acked Read 3 Done Sent Done false
7908 2781 Acked Read 9 Done Sent Done false
7909 2782 Acked Write 2 Done Sent Done false
7910 2783 Acked Read 5 Done Sent Done false
7911 2784 Acked Read 9 Done Sent Done false
7912 2785 Acked Write 9 Done Sent Done false
7913 2786 Acked Write 3 Done Sent Done false
7914 2787 Acked Write 5 Done Sent Done false
7915 2788 Acked Write 8 Done Sent Done false
7916 2789 Acked Read 7 Done Sent Done false
7917 2790 Acked Read 5 Done Sent Done false
7918 2791 Acked Write 5 Done Sent Done false
7919 2792 Acked Write 1 Done Sent Done false
7920 2793 Acked Flush 0 Done Sent Done false
7921 2794 Acked Write 4 Done Sent Done false
7922 2795 Acked Flush 0 Done Sent Done false
7923 2796 Acked Write 6 Done Sent Done false
7924 2797 Acked Write 3 Done Sent Done false
7925 2798 Acked Read 3 Done Sent Done false
7926 2799 Acked Read 10 Done Sent Done false
7927 2800 Acked Read 5 Done Sent Done false
7928 2801 Acked Read 2 Done Sent Done false
7929 2802 Acked Write 9 Done Sent Done false
7930 2803 Acked Flush 0 Done Sent Done false
7931 2804 Acked Read 3 Done Sent Done false
7932 2805 Acked Read 8 Done Sent Done false
7933 2806 Acked Write 2 Done Sent Done false
7934 2807 Acked Write 10 Done Sent Done false
7935 2808 Acked Write 10 Done Sent Done false
7936 2809 Acked Write 9 Done Sent Done false
7937 2810 Acked Read 4 Done Sent Done false
7938 2811 Acked Read 10 Done Sent Done false
7939 2812 Acked Write 7 Done Sent Done false
7940 2813 Acked Read 8 Done Sent Done false
7941 2814 Acked Flush 0 Done Sent Done false
7942 STATES DS:0 DS:1 DS:2 TOTAL
7943 Sent 0 303 0 303
7944 Done 303 0 303 606
7945 Skipped 0 0 0 0
7946 Error 0 0 0 0
7947 Last Flush: 2814 2511 2814
7948 Downstairs last five completed: 2511 2510 2509 2508 2507
7949 Upstairs last five completed: 2814 2813 2812 2811 2810
7950 CLIENT: Up:0 ds:303 act:2
79512024-11-07T14:04:59.280ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79522024-11-07T14:04:59.280ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62899) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79532024-11-07T14:04:59.281ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79542024-11-07T14:04:59.281ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79552024-11-07T14:04:59.281ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79562024-11-07T14:04:59.281ZINFOcrucible: send last flush ID to this DS: 2511 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79572024-11-07T14:04:59.281ZINFOcrucible: Replied this last flush ID: 2511 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79582024-11-07T14:04:59.282ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79592024-11-07T14:04:59.282ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79602024-11-07T14:04:59.282ZINFOcrucible: [1] client re-new 303 jobs since flush 2511 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79612024-11-07T14:04:59.282ZINFOcrucible: [1] Replayed 303 jobs since flush: 2511 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
7962 ----------------------------------------------------------------
7963 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7964 Downstairs last five completed: 2814 2813 2812 2811 2810
7965 Upstairs last five completed: 2814 2813 2812 2811 2810
7966 CLIENT: Up:0 ds:0 act:3
7967 Replay: All jobs finished
7968 Main task has work to do, go find it
7969 got dsc Stop(0)
7970 stop 0
7971 Nov 07 14:04:59.581 [8810] Got stop action so:false kr:false
7972 INFO request completed, latency_us: 348, response_code: 204, uri: /stop/cid/0, method: POST, req_id: ba8a0123-b6e5-43d5-b933-fa587c5c61bc, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
79732024-11-07T14:04:59.581ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79742024-11-07T14:04:59.581ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79752024-11-07T14:04:59.581ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79762024-11-07T14:04:59.581ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79772024-11-07T14:04:59.581ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
79782024-11-07T14:04:59.582ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
7979 Nov 07 14:04:59.582 INFO request completed, latency_us: 159, response_code: 200, uri: /state/cid/0, method: GET, req_id: 005bd8d3-5bd5-4bd3-aae3-54d69d672b6f, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
7980 [8810] Exited with: ExitStatus(unix_wait_status(9))
7981 [8810][0] reports Exit
7982 Nov 07 14:05:03.584 INFO request completed, latency_us: 254, response_code: 200, uri: /state/cid/0, method: GET, req_id: 6f483bbf-6080-44d5-999e-60610f687225, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
7983 001/300 Write block 415 len 24576 data: 2 2 2 2 2 3
7984 002/300 Read block 3771 len 36864
7985 003/300 Write block 1134 len 4096 data: 2
7986 004/300 Write block 72 len 8192 data: 4 4
7987 005/300 Read block 2357 len 8192
7988 006/300 Read block 1667 len 28672
7989 007/300 Read block 580 len 32768
7990 008/300 Read block 2492 len 16384
7991 009/300 Read block 3865 len 40960
7992 010/300 Read block 3465 len 20480
7993 011/300 Write block 4280 len 8192 data: 3 3
7994 012/300 Flush
7995 013/300 Read block 4752 len 8192
7996 014/300 Write block 3 len 16384 data: 2 2 2 2
7997 015/300 Write block 3445 len 4096 data: 3
7998 016/300 Read block 3774 len 12288
7999 017/300 Read block 2426 len 16384
8000 018/300 Write block 189 len 16384 data: 3 3 2 3
8001 019/300 Write block 55 len 4096 data: 4
8002 020/300 Write block 1649 len 36864 data: 2 2 2 2 2 2 2 2 2
8003 021/300 Write block 3336 len 20480 data: 4 3 3 3 3
8004 022/300 Read block 1178 len 8192
8005 023/300 Write block 4931 len 20480 data: 2 2 3 3 3
8006 024/300 Read block 3661 len 12288
8007 025/300 Write block 3488 len 28672 data: 4 2 2 2 3 2 2
8008 026/300 Write block 2226 len 36864 data: 3 3 3 3 3 3 3 3 2
8009 027/300 Read block 453 len 20480
8010 028/300 Read block 3563 len 12288
8011 029/300 Read block 272 len 36864
8012 030/300 Write block 4311 len 36864 data: 2 2 3 4 4 4 4 4 4
8013 031/300 Write block 635 len 8192 data: 3 3
8014 032/300 Write block 1981 len 40960 data: 2 2 2 3 3 3 3 3 2 2
8015 033/300 Write block 3477 len 32768 data: 4 4 4 4 3 3 4 4
8016 034/300 Read block 3071 len 4096
8017 035/300 Read block 4236 len 32768
8018 036/300 Write block 896 len 40960 data: 5 3 4 5 4 4 3 3 3 3
8019 037/300 Write block 146 len 8192 data: 2 2
8020 038/300 Read block 2180 len 4096
8021 039/300 Read block 764 len 16384
8022 040/300 Read block 2504 len 16384
8023 041/300 Flush
8024 042/300 Write block 3005 len 32768 data: 2 2 2 2 2 2 3 3
8025 043/300 Flush
8026 044/300 Read block 3208 len 16384
8027 045/300 Write block 1675 len 32768 data: 5 4 3 4 4 4 4 3
8028 046/300 Write block 2335 len 4096 data: 3
8029 047/300 Write block 111 len 8192 data: 2 2
8030 048/300 Write block 4785 len 8192 data: 2 2
8031 049/300 Flush
8032 050/300 Write block 4506 len 40960 data: 2 2 2 2 2 2 2 2 2 2
8033 051/300 Read block 4272 len 12288
8034 052/300 Read block 2923 len 32768
8035 053/300 Read block 3367 len 8192
8036 054/300 Read block 3265 len 24576
8037 055/300 Write block 1667 len 4096 data: 2
8038 056/300 Read block 1770 len 20480
8039 057/300 Read block 701 len 24576
8040 058/300 Read block 3846 len 32768
8041 059/300 Read block 3292 len 36864
8042 060/300 Write block 1660 len 32768 data: 2 2 2 2 2 2 2 3
8043 061/300 Read block 359 len 12288
8044 062/300 Read block 38 len 24576
8045 063/300 Write block 1647 len 36864 data: 4 4 3 3 3 3 3 3 3
8046 064/300 Write block 4959 len 40960 data: 3 3 2 2 2 2 3 3 3 3
8047 065/300 Write block 1666 len 36864 data: 3 4 2 2 2 2 2 3 3
8048 066/300 Read block 4647 len 4096
8049 067/300 Read block 232 len 8192
8050 068/300 Write block 1124 len 28672 data: 3 3 3 2 2 2 2
8051 069/300 Write block 3438 len 40960 data: 2 3 4 4 4 4 3 4 4 3
8052 070/300 Flush
8053 071/300 Write block 1046 len 40960 data: 3 3 3 3 3 3 3 3 2 2
8054 072/300 Read block 2717 len 28672
8055 073/300 Read block 4010 len 4096
8056 074/300 Read block 567 len 40960
8057 075/300 Read block 3635 len 24576
8058 076/300 Read block 2461 len 32768
8059 077/300 Read block 3682 len 16384
8060 078/300 Write block 4573 len 36864 data: 3 4 3 3 3 3 3 2 2
8061 079/300 Flush
8062 080/300 Write block 4532 len 32768 data: 3 2 3 3 2 2 2 2
8063 081/300 Read block 2098 len 20480
8064 082/300 Read block 4393 len 16384
8065 083/300 Write block 956 len 12288 data: 5 4 2
8066 084/300 Read block 3357 len 28672
8067 085/300 Write block 4727 len 20480 data: 3 3 3 3 3
8068 086/300 Read block 3623 len 4096
8069 087/300 Read block 2913 len 12288
8070 088/300 Write block 2181 len 20480 data: 4 4 4 5 5
8071 089/300 Flush
8072 090/300 Write block 1548 len 12288 data: 3 2 2
8073 091/300 Write block 4507 len 32768 data: 3 3 3 3 3 3 3 3
8074 092/300 Read block 2983 len 4096
8075 093/300 Flush
8076 094/300 Write block 2631 len 12288 data: 2 2 2
8077 095/300 Write block 4037 len 40960 data: 2 2 2 2 3 3 3 4 4 3
8078 096/300 Write block 2057 len 20480 data: 3 2 3 3 3
8079 097/300 Write block 3168 len 24576 data: 3 2 2 2 2 2
8080 098/300 Read block 4003 len 32768
8081 099/300 Write block 3088 len 16384 data: 4 4 4 4
8082 100/300 Read block 2217 len 16384
8083 101/300 Write block 1837 len 28672 data: 2 2 2 2 3 3 3
8084 102/300 Read block 709 len 4096
8085 103/300 Write block 3401 len 12288 data: 4 4 4
8086 104/300 Read block 1142 len 40960
8087 105/300 Write block 3731 len 36864 data: 4 4 3 3 4 4 4 4 4
8088 106/300 Write block 1248 len 32768 data: 2 3 2 2 2 2 3 3
8089 107/300 Flush
8090 108/300 Read block 506 len 4096
8091 109/300 Read block 1202 len 28672
8092 110/300 Write block 2085 len 8192 data: 2 2
8093 111/300 Write block 207 len 32768 data: 2 2 2 2 2 2 2 2
8094 112/300 Write block 4238 len 8192 data: 3 3
8095 113/300 Read block 3204 len 20480
8096 114/300 Write block 1347 len 16384 data: 2 2 3 3
8097 115/300 Read block 3740 len 8192
8098 116/300 Write block 1586 len 8192 data: 2 2
8099 117/300 Read block 1838 len 8192
8100 118/300 Read block 1505 len 24576
8101 119/300 Read block 4277 len 36864
8102 120/300 Flush
8103 121/300 Read block 2486 len 40960
8104 122/300 Write block 2270 len 32768 data: 3 3 3 3 3 3 3 4
8105 123/300 Write block 1333 len 24576 data: 3 2 2 2 2 2
8106 124/300 Read block 4077 len 40960
8107 125/300 Write block 1659 len 8192 data: 2 3
8108 126/300 Read block 1889 len 28672
8109 127/300 Flush
8110 128/300 Write block 906 len 32768 data: 3 2 2 2 2 2 2 2
8111 129/300 Write block 1191 len 28672 data: 4 4 3 3 3 3 3
8112 130/300 Write block 939 len 36864 data: 2 2 2 2 2 2 2 2 2
8113 131/300 Read block 2513 len 8192
8114 132/300 Read block 2902 len 24576
8115 133/300 Read block 3294 len 28672
8116 134/300 Write block 1072 len 40960 data: 3 3 3 3 3 3 4 4 5 5
8117 135/300 Read block 2535 len 24576
8118 136/300 Write block 3863 len 32768 data: 2 3 3 2 2 2 2 3
8119 137/300 Write block 1791 len 20480 data: 3 3 3 4 4
8120 138/300 Flush
8121 139/300 Read block 3811 len 20480
8122 140/300 Read block 4124 len 12288
8123 141/300 Read block 1467 len 40960
8124 142/300 Read block 4447 len 36864
8125 143/300 Read block 4057 len 16384
8126 144/300 Write block 872 len 32768 data: 2 2 2 2 2 2 2 2
8127 145/300 Read block 2194 len 24576
8128 146/300 Read block 4434 len 8192
8129 147/300 Write block 1 len 16384 data: 2 2 3 3
8130 148/300 Flush
8131 149/300 Flush
8132 150/300 Read block 4550 len 36864
8133 151/300 Write block 3289 len 16384 data: 3 3 2 2
8134 152/300 Flush
8135 153/300 Write block 4765 len 12288 data: 2 2 2
8136 154/300 Write block 3700 len 24576 data: 2 2 2 3 3 3
8137 155/300 Read block 2855 len 36864
8138 156/300 Read block 4361 len 36864
8139 157/300 Read block 4213 len 8192
8140 158/300 Read block 1377 len 16384
8141 159/300 Flush
8142 160/300 Read block 1840 len 16384
8143 161/300 Read block 4937 len 32768
8144 162/300 Write block 4347 len 28672 data: 2 2 2 2 2 2 2
8145 163/300 Read block 656 len 8192
8146 164/300 Read block 1094 len 16384
8147 165/300 Write block 1997 len 24576 data: 2 2 2 2 2 3
8148 166/300 Flush
8149 167/300 Write block 4641 len 40960 data: 2 2 2 2 2 2 2 2 2 2
8150 168/300 Read block 1809 len 32768
8151 169/300 Write block 605 len 8192 data: 3 3
8152 170/300 Read block 495 len 36864
8153 171/300 Flush
8154 172/300 Read block 4988 len 32768
8155 173/300 Read block 3247 len 24576
8156 174/300 Read block 4376 len 12288
8157 175/300 Write block 2488 len 4096 data: 3
8158 176/300 Flush
8159 177/300 Read block 2301 len 28672
8160 178/300 Write block 1171 len 32768 data: 2 2 2 2 2 2 2 2
8161 179/300 Flush
8162 180/300 Write block 4647 len 8192 data: 3 3
8163 181/300 Read block 100 len 20480
8164 182/300 Read block 3871 len 8192
8165 183/300 Read block 966 len 28672
8166 184/300 Read block 77 len 8192
8167 185/300 Read block 3751 len 4096
8168 186/300 Read block 1304 len 4096
8169 187/300 Read block 3308 len 32768
8170 188/300 Read block 1261 len 40960
8171 189/300 Read block 4491 len 28672
8172 190/300 Read block 1036 len 16384
8173 191/300 Write block 3912 len 16384 data: 4 3 3 3
8174 192/300 Write block 1811 len 16384 data: 4 3 2 2
8175 193/300 Read block 4443 len 40960
8176 194/300 Write block 1374 len 12288 data: 3 3 3
8177 195/300 Write block 2470 len 28672 data: 2 2 3 3 3 4 4
8178 196/300 Read block 584 len 28672
8179 197/300 Read block 1172 len 20480
8180 198/300 Read block 3214 len 4096
8181 199/300 Write block 4706 len 12288 data: 2 2 2
8182 200/300 Read block 3208 len 36864
8183 201/300 Read block 1 len 40960
8184 202/300 Read block 3633 len 8192
8185 203/300 Flush
8186 204/300 Read block 4370 len 28672
8187 205/300 Read block 3950 len 28672
8188 206/300 Write block 3380 len 32768 data: 3 3 2 2 2 2 2 3
8189 207/300 Read block 3702 len 40960
8190 208/300 Flush
8191 209/300 Flush
8192 210/300 Flush
8193 211/300 Read block 3986 len 16384
8194 212/300 Read block 3978 len 12288
8195 213/300 Read block 4438 len 20480
8196 214/300 Write block 1847 len 40960 data: 2 2 2 4 3 4 5 5 4 4
8197 215/300 Flush
8198 216/300 Write block 1052 len 20480 data: 4 4 3 3 2
8199 217/300 Write block 1789 len 16384 data: 3 3 4 4
8200 218/300 Write block 4446 len 20480 data: 2 3 3 3 3
8201 219/300 Read block 2324 len 16384
8202 220/300 Flush
8203 221/300 Flush
8204 222/300 Read block 477 len 16384
8205 223/300 Write block 3014 len 16384 data: 2 2 2 2
8206 224/300 Flush
8207 225/300 Read block 4226 len 8192
8208 226/300 Write block 2576 len 20480 data: 3 3 3 3 3
8209 227/300 Write block 1816 len 40960 data: 2 2 2 2 2 2 2 2 2 2
8210 228/300 Read block 2501 len 40960
8211 229/300 Read block 3978 len 8192
8212 230/300 Write block 1588 len 24576 data: 2 2 2 2 2 2
8213 231/300 Read block 3468 len 28672
8214 232/300 Flush
8215 233/300 Write block 3448 len 16384 data: 3 2 2 2
8216 234/300 Write block 3422 len 24576 data: 2 2 5 5 5 5
8217 235/300 Write block 3598 len 40960 data: 3 3 3 3 3 3 4 3 3 3
8218 236/300 Read block 3142 len 16384
8219 237/300 Read block 696 len 4096
8220 238/300 Write block 1167 len 32768 data: 2 2 2 2 3 3 3 3
8221 239/300 Write block 2745 len 4096 data: 4
8222 240/300 Write block 3214 len 4096 data: 3
8223 241/300 Write block 1914 len 36864 data: 2 2 2 2 2 3 3 3 3
8224 242/300 Read block 2466 len 28672
8225 243/300 Read block 1251 len 12288
8226 244/300 Read block 4446 len 16384
8227 245/300 Write block 1667 len 16384 data: 5 3 3 3
8228 246/300 Write block 3105 len 4096 data: 3
8229 247/300 Read block 4836 len 8192
8230 248/300 Read block 2992 len 12288
8231 249/300 Flush
8232 250/300 Write block 992 len 28672 data: 2 2 2 2 2 2 2
8233 251/300 Read block 3395 len 4096
8234 252/300 Read block 3237 len 8192
8235 253/300 Read block 2824 len 4096
8236 254/300 Flush
8237 255/300 Write block 3813 len 16384 data: 3 4 4 4
8238 256/300 Read block 3186 len 36864
8239 257/300 Read block 462 len 36864
8240 258/300 Flush
8241 259/300 Write block 4345 len 32768 data: 2 2 3 3 3 3 3 3
8242 260/300 Read block 4221 len 12288
8243 261/300 Write block 1271 len 20480 data: 2 2 2 2 2
8244 262/300 Write block 4475 len 32768 data: 3 3 3 3 3 3 3 3
8245 263/300 Write block 4870 len 32768 data: 2 2 2 2 2 2 2 2
8246 264/300 Read block 1738 len 20480
8247 265/300 Read block 4199 len 28672
8248 266/300 Read block 876 len 12288
8249 267/300 Write block 4227 len 40960 data: 2 2 2 2 2 2 2 2 2 2
8250 268/300 Read block 1444 len 32768
8251 269/300 Write block 1193 len 40960 data: 4 4 4 4 4 2 2 2 2 2
8252 270/300 Write block 870 len 36864 data: 3 2 3 3 3 3 3 3 3
8253 271/300 Read block 715 len 8192
8254 272/300 Write block 3219 len 12288 data: 3 3 2
8255 273/300 Read block 1351 len 20480
8256 274/300 Write block 3067 len 4096 data: 2
8257 275/300 Read block 1340 len 32768
8258 276/300 Read block 3590 len 28672
8259 277/300 Flush
8260 278/300 Read block 38 len 36864
8261 279/300 Read block 225 len 8192
8262 280/300 Read block 580 len 4096
8263 281/300 Flush
8264 282/300 Read block 402 len 12288
8265 283/300 Read block 2713 len 4096
8266 284/300 Read block 3811 len 24576
8267 285/300 Write block 1713 len 36864 data: 2 2 2 2 2 2 2 3 2
8268 286/300 Read block 4343 len 32768
8269 287/300 Read block 1377 len 12288
8270 288/300 Read block 4848 len 32768
8271 289/300 Write block 4337 len 28672 data: 4 4 3 3 3 3 2
8272 290/300 Read block 552 len 32768
8273 291/300 Read block 273 len 36864
8274 292/300 Read block 4337 len 32768
8275 293/300 Read block 4958 len 8192
8276 294/300 Write block 903 len 28672 data: 4 4 4 4 3 3 3
8277 295/300 Write block 2432 len 36864 data: 2 2 2 2 2 4 4 4 4
8278 296/300 Write block 3531 len 20480 data: 2 2 2 2 2
8279 297/300 Read block 3152 len 16384
8280 298/300 Read block 4310 len 4096
8281 299/300 Write block 3068 len 8192 data: 2 2
8282 300/300 Flush
8283 301/300 Read block 2551 len 36864
8284 Main task has work to do, go find it
8285 got dsc Start(0)
8286 start 0
8287 Nov 07 14:05:03.870 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/0, method: POST, req_id: 243d0a83-9cd6-403e-83d2-8770cf41bdcb, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
8288 Replay: started 0, returned:Ok(())
8289 ----------------------------------------------------------------
8290 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8291 States: Offline Active Active
8292 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8293 2815 Acked Write 6 Sent Done Done false
8294 2816 Acked Read 9 Sent Done Done false
8295 2817 Acked Write 1 Sent Done Done false
8296 2818 Acked Write 2 Sent Done Done false
8297 2819 Acked Read 2 Sent Done Done false
8298 2820 Acked Read 7 Sent Done Done false
8299 2821 Acked Read 8 Sent Done Done false
8300 2822 Acked Read 4 Sent Done Done false
8301 2823 Acked Read 10 Sent Done Done false
8302 2824 Acked Read 5 Sent Done Done false
8303 2825 Acked Write 2 Sent Done Done false
8304 2826 Acked Flush 0 Sent Done Done false
8305 2827 Acked Read 2 Sent Done Done false
8306 2828 Acked Write 4 Sent Done Done false
8307 2829 Acked Write 1 Sent Done Done false
8308 2830 Acked Read 3 Sent Done Done false
8309 2831 Acked Read 4 Sent Done Done false
8310 2832 Acked Write 4 Sent Done Done false
8311 2833 Acked Write 1 Sent Done Done false
8312 2834 Acked Write 9 Sent Done Done false
8313 2835 Acked Write 5 Sent Done Done false
8314 2836 Acked Read 2 Sent Done Done false
8315 2837 Acked Write 5 Sent Done Done false
8316 2838 Acked Read 3 Sent Done Done false
8317 2839 Acked Write 7 Sent Done Done false
8318 2840 Acked Write 9 Sent Done Done false
8319 2841 Acked Read 5 Sent Done Done false
8320 2842 Acked Read 3 Sent Done Done false
8321 2843 Acked Read 9 Sent Done Done false
8322 2844 Acked Write 9 Sent Done Done false
8323 2845 Acked Write 2 Sent Done Done false
8324 2846 Acked Write 10 Sent Done Done false
8325 2847 Acked Write 8 Sent Done Done false
8326 2848 Acked Read 1 Sent Done Done false
8327 2849 Acked Read 8 Sent Done Done false
8328 2850 Acked Write 10 Sent Done Done false
8329 2851 Acked Write 2 Sent Done Done false
8330 2852 Acked Read 1 Sent Done Done false
8331 2853 Acked Read 4 Sent Done Done false
8332 2854 Acked Read 4 Sent Done Done false
8333 2855 Acked Flush 0 Sent Done Done false
8334 2856 Acked Write 8 Sent Done Done false
8335 2857 Acked Flush 0 Sent Done Done false
8336 2858 Acked Read 4 Sent Done Done false
8337 2859 Acked Write 8 Sent Done Done false
8338 2860 Acked Write 1 Sent Done Done false
8339 2861 Acked Write 2 Sent Done Done false
8340 2862 Acked Write 2 Sent Done Done false
8341 2863 Acked Flush 0 Sent Done Done false
8342 2864 Acked Write 10 Sent Done Done false
8343 2865 Acked Read 3 Sent Done Done false
8344 2866 Acked Read 8 Sent Done Done false
8345 2867 Acked Read 2 Sent Done Done false
8346 2868 Acked Read 6 Sent Done Done false
8347 2869 Acked Write 1 Sent Done Done false
8348 2870 Acked Read 5 Sent Done Done false
8349 2871 Acked Read 6 Sent Done Done false
8350 2872 Acked Read 8 Sent Done Done false
8351 2873 Acked Read 9 Sent Done Done false
8352 2874 Acked Write 8 Sent Done Done false
8353 2875 Acked Read 3 Sent Done Done false
8354 2876 Acked Read 6 Sent Done Done false
8355 2877 Acked Write 9 Sent Done Done false
8356 2878 Acked Write 10 Sent Done Done false
8357 2879 Acked Write 9 Sent Done Done false
8358 2880 Acked Read 1 Sent Done Done false
8359 2881 Acked Read 2 Sent Done Done false
8360 2882 Acked Write 7 Sent Done Done false
8361 2883 Acked Write 10 Sent Done Done false
8362 2884 Acked Flush 0 Sent Done Done false
8363 2885 Acked Write 10 Sent Done Done false
8364 2886 Acked Read 7 Sent Done Done false
8365 2887 Acked Read 1 Sent Done Done false
8366 2888 Acked Read 10 Sent Done Done false
8367 2889 Acked Read 6 Sent Done Done false
8368 2890 Acked Read 8 Sent Done Done false
8369 2891 Acked Read 4 Sent Done Done false
8370 2892 Acked Write 9 Sent Done Done false
8371 2893 Acked Flush 0 Sent Done Done false
8372 2894 Acked Write 8 Sent Done Done false
8373 2895 Acked Read 5 Sent Done Done false
8374 2896 Acked Read 4 Sent Done Done false
8375 2897 Acked Write 3 Sent Done Done false
8376 2898 Acked Read 7 Sent Done Done false
8377 2899 Acked Write 5 Sent Done Done false
8378 2900 Acked Read 1 Sent Done Done false
8379 2901 Acked Read 3 Sent Done Done false
8380 2902 Acked Write 5 Sent Done Done false
8381 2903 Acked Flush 0 Sent Done Done false
8382 2904 Acked Write 3 Sent Done Done false
8383 2905 Acked Write 8 Sent Done Done false
8384 2906 Acked Read 1 Sent Done Done false
8385 2907 Acked Flush 0 Sent Done Done false
8386 2908 Acked Write 3 Sent Done Done false
8387 2909 Acked Write 10 Sent Done Done false
8388 2910 Acked Write 5 Sent Done Done false
8389 2911 Acked Write 6 Sent Done Done false
8390 2912 Acked Read 8 Sent Done Done false
8391 2913 Acked Write 4 Sent Done Done false
8392 2914 Acked Read 4 Sent Done Done false
8393 2915 Acked Write 7 Sent Done Done false
8394 2916 Acked Read 1 Sent Done Done false
8395 2917 Acked Write 3 Sent Done Done false
8396 2918 Acked Read 10 Sent Done Done false
8397 2919 Acked Write 9 Sent Done Done false
8398 2920 Acked Write 8 Sent Done Done false
8399 2921 Acked Flush 0 Sent Done Done false
8400 2922 Acked Read 1 Sent Done Done false
8401 2923 Acked Read 7 Sent Done Done false
8402 2924 Acked Write 2 Sent Done Done false
8403 2925 Acked Write 8 Sent Done Done false
8404 2926 Acked Write 2 Sent Done Done false
8405 2927 Acked Read 5 Sent Done Done false
8406 2928 Acked Write 4 Sent Done Done false
8407 2929 Acked Read 2 Sent Done Done false
8408 2930 Acked Write 2 Sent Done Done false
8409 2931 Acked Read 2 Sent Done Done false
8410 2932 Acked Read 6 Sent Done Done false
8411 2933 Acked Read 9 Sent Done Done false
8412 2934 Acked Flush 0 Sent Done Done false
8413 2935 Acked Read 10 Sent Done Done false
8414 2936 Acked Write 8 Sent Done Done false
8415 2937 Acked Write 6 Sent Done Done false
8416 2938 Acked Read 10 Sent Done Done false
8417 2939 Acked Write 2 Sent Done Done false
8418 2940 Acked Read 7 Sent Done Done false
8419 2941 Acked Flush 0 Sent Done Done false
8420 2942 Acked Write 8 Sent Done Done false
8421 2943 Acked Write 7 Sent Done Done false
8422 2944 Acked Write 9 Sent Done Done false
8423 2945 Acked Read 2 Sent Done Done false
8424 2946 Acked Read 6 Sent Done Done false
8425 2947 Acked Read 7 Sent Done Done false
8426 2948 Acked Write 10 Sent Done Done false
8427 2949 Acked Read 6 Sent Done Done false
8428 2950 Acked Write 8 Sent Done Done false
8429 2951 Acked Write 5 Sent Done Done false
8430 2952 Acked Flush 0 Sent Done Done false
8431 2953 Acked Read 5 Sent Done Done false
8432 2954 Acked Read 3 Sent Done Done false
8433 2955 Acked Read 10 Sent Done Done false
8434 2956 Acked Read 9 Sent Done Done false
8435 2957 Acked Read 4 Sent Done Done false
8436 2958 Acked Write 8 Sent Done Done false
8437 2959 Acked Read 6 Sent Done Done false
8438 2960 Acked Read 2 Sent Done Done false
8439 2961 Acked Write 4 Sent Done Done false
8440 2962 Acked Flush 0 Sent Done Done false
8441 2963 Acked Flush 0 Sent Done Done false
8442 2964 Acked Read 9 Sent Done Done false
8443 2965 Acked Write 4 Sent Done Done false
8444 2966 Acked Flush 0 Sent Done Done false
8445 2967 Acked Write 3 Sent Done Done false
8446 2968 Acked Write 6 Sent Done Done false
8447 2969 Acked Read 9 Sent Done Done false
8448 2970 Acked Read 9 Sent Done Done false
8449 2971 Acked Read 2 Sent Done Done false
8450 2972 Acked Read 4 Sent Done Done false
8451 2973 Acked Flush 0 Sent Done Done false
8452 2974 Acked Read 4 Sent Done Done false
8453 2975 Acked Read 8 Sent Done Done false
8454 2976 Acked Write 7 Sent Done Done false
8455 2977 Acked Read 2 Sent Done Done false
8456 2978 Acked Read 4 Sent Done Done false
8457 2979 Acked Write 6 Sent Done Done false
8458 2980 Acked Flush 0 Sent Done Done false
8459 2981 Acked Write 10 Sent Done Done false
8460 2982 Acked Read 8 Sent Done Done false
8461 2983 Acked Write 2 Sent Done Done false
8462 2984 Acked Read 9 Sent Done Done false
8463 2985 Acked Flush 0 Sent Done Done false
8464 2986 Acked Read 8 Sent Done Done false
8465 2987 Acked Read 6 Sent Done Done false
8466 2988 Acked Read 3 Sent Done Done false
8467 2989 Acked Write 1 Sent Done Done false
8468 2990 Acked Flush 0 Sent Done Done false
8469 2991 Acked Read 7 Sent Done Done false
8470 2992 Acked Write 8 Sent Done Done false
8471 2993 Acked Flush 0 Sent Done Done false
8472 2994 Acked Write 2 Sent Done Done false
8473 2995 Acked Read 5 Sent Done Done false
8474 2996 Acked Read 2 Sent Done Done false
8475 2997 Acked Read 7 Sent Done Done false
8476 2998 Acked Read 2 Sent Done Done false
8477 2999 Acked Read 1 Sent Done Done false
8478 3000 Acked Read 1 Sent Done Done false
8479 3001 Acked Read 8 Sent Done Done false
8480 3002 Acked Read 10 Sent Done Done false
8481 3003 Acked Read 7 Sent Done Done false
8482 3004 Acked Read 4 Sent Done Done false
8483 3005 Acked Write 4 Sent Done Done false
8484 3006 Acked Write 4 Sent Done Done false
8485 3007 Acked Read 10 Sent Done Done false
8486 3008 Acked Write 3 Sent Done Done false
8487 3009 Acked Write 7 Sent Done Done false
8488 3010 Acked Read 7 Sent Done Done false
8489 3011 Acked Read 5 Sent Done Done false
8490 3012 Acked Read 1 Sent Done Done false
8491 3013 Acked Write 3 Sent Done Done false
8492 3014 Acked Read 9 Sent Done Done false
8493 3015 Acked Read 10 Sent Done Done false
8494 3016 Acked Read 2 Sent Done Done false
8495 3017 Acked Flush 0 Sent Done Done false
8496 3018 Acked Read 7 Sent Done Done false
8497 3019 Acked Read 7 Sent Done Done false
8498 3020 Acked Write 8 Sent Done Done false
8499 3021 Acked Read 10 Sent Done Done false
8500 3022 Acked Flush 0 Sent Done Done false
8501 3023 Acked Flush 0 Sent Done Done false
8502 3024 Acked Flush 0 Sent Done Done false
8503 3025 Acked Read 4 Sent Done Done false
8504 3026 Acked Read 3 Sent Done Done false
8505 3027 Acked Read 5 Sent Done Done false
8506 3028 Acked Write 10 Sent Done Done false
8507 3029 Acked Flush 0 Sent Done Done false
8508 3030 Acked Write 5 Sent Done Done false
8509 3031 Acked Write 4 Sent Done Done false
8510 3032 Acked Write 5 Sent Done Done false
8511 3033 Acked Read 4 Sent Done Done false
8512 3034 Acked Flush 0 Sent Done Done false
8513 3035 Acked Flush 0 Sent Done Done false
8514 3036 Acked Read 4 Sent Done Done false
8515 3037 Acked Write 4 Sent Done Done false
8516 3038 Acked Flush 0 Sent Done Done false
8517 3039 Acked Read 2 Sent Done Done false
8518 3040 Acked Write 5 Sent Done Done false
8519 3041 Acked Write 10 Sent Done Done false
8520 3042 Acked Read 10 Sent Done Done false
8521 3043 Acked Read 2 Sent Done Done false
8522 3044 Acked Write 6 Sent Done Done false
8523 3045 Acked Read 7 Sent Done Done false
8524 3046 Acked Flush 0 Sent Done Done false
8525 3047 Acked Write 4 Sent Done Done false
8526 3048 Acked Write 6 Sent Done Done false
8527 3049 Acked Write 10 Sent Done Done false
8528 3050 Acked Read 4 Sent Done Done false
8529 3051 Acked Read 1 Sent Done Done false
8530 3052 Acked Write 8 Sent Done Done false
8531 3053 Acked Write 1 Sent Done Done false
8532 3054 Acked Write 1 Sent Done Done false
8533 3055 Acked Write 9 Sent Done Done false
8534 3056 Acked Read 7 Sent Done Done false
8535 3057 Acked Read 3 Sent Done Done false
8536 3058 Acked Read 4 Sent Done Done false
8537 3059 Acked Write 4 Sent Done Done false
8538 3060 Acked Write 1 Sent Done Done false
8539 3061 Acked Read 2 Sent Done Done false
8540 3062 Acked Read 3 Sent Done Done false
8541 3063 Acked Flush 0 Sent Done Done false
8542 3064 Acked Write 7 Sent Done Done false
8543 3065 Acked Read 1 Sent Done Done false
8544 3066 Acked Read 2 Sent Done Done false
8545 3067 Acked Read 1 Sent Done Done false
8546 3068 Acked Flush 0 Sent Done Done false
8547 3069 Acked Write 4 Sent Done Done false
8548 3070 Acked Read 9 Sent Done Done false
8549 3071 Acked Read 9 Sent Done Done false
8550 3072 Acked Flush 0 Sent Done Done false
8551 3073 Acked Write 8 Sent Done Done false
8552 3074 Acked Read 3 Sent Done Done false
8553 3075 Acked Write 5 Sent Done Done false
8554 3076 Acked Write 8 Sent Done Done false
8555 3077 Acked Write 8 Sent Done Done false
8556 3078 Acked Read 5 Sent Done Done false
8557 3079 Acked Read 7 Sent Done Done false
8558 3080 Acked Read 3 Sent Done Done false
8559 3081 Acked Write 10 Sent Done Done false
8560 3082 Acked Read 8 Sent Done Done false
8561 3083 Acked Write 10 Sent Done Done false
8562 3084 Acked Write 9 Sent Done Done false
8563 3085 Acked Read 2 Sent Done Done false
8564 3086 Acked Write 3 Sent Done Done false
8565 3087 Acked Read 5 Sent Done Done false
8566 3088 Acked Write 1 Sent Done Done false
8567 3089 Acked Read 8 Sent Done Done false
8568 3090 Acked Read 7 Sent Done Done false
8569 3091 Acked Flush 0 Sent Done Done false
8570 3092 Acked Read 9 Sent Done Done false
8571 3093 Acked Read 2 Sent Done Done false
8572 3094 Acked Read 1 Sent Done Done false
8573 3095 Acked Flush 0 Sent Done Done false
8574 3096 Acked Read 3 Sent Done Done false
8575 3097 Acked Read 1 Sent Done Done false
8576 3098 Acked Read 6 Sent Done Done false
8577 3099 Acked Write 9 Sent Done Done false
8578 3100 Acked Read 8 Sent Done Done false
8579 3101 Acked Read 3 Sent Done Done false
8580 3102 Acked Read 8 Sent Done Done false
8581 3103 Acked Write 7 Sent Done Done false
8582 3104 Acked Read 8 Sent Done Done false
8583 3105 Acked Read 9 Sent Done Done false
8584 3106 Acked Read 8 Sent Done Done false
8585 3107 Acked Read 2 Sent Done Done false
8586 3108 Acked Write 7 Sent Done Done false
8587 3109 Acked Write 9 Sent Done Done false
8588 3110 Acked Write 5 Sent Done Done false
8589 3111 Acked Read 4 Sent Done Done false
8590 3112 Acked Read 1 Sent Done Done false
8591 3113 Acked Write 2 Sent Done Done false
8592 3114 Acked Flush 0 Sent Done Done false
8593 3115 Acked Read 9 Sent Done Done false
8594 STATES DS:0 DS:1 DS:2 TOTAL
8595 Sent 301 0 0 301
8596 Done 0 301 301 602
8597 Skipped 0 0 0 0
8598 Error 0 0 0 0
8599 Last Flush: 2814 3114 3114
8600 Downstairs last five completed: 2814 2813 2812 2811 2810
8601 Upstairs last five completed: 3115 3114 3113 3112 3111
8602 CLIENT: Up:0 ds:301 act:2
8603 [8810] got start action
8604 [8810] I am going to restart
8605 Starting downstairs at port 8810
8606 Make output file at "/tmp/dsc/downstairs-8810.txt"
8607 [8810][0] reports Starting
8608 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1206)
8609 [8810][0] reports Running
8610 ----------------------------------------------------------------
8611 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8612 States: Offline Active Active
8613 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8614 2815 Acked Write 6 Sent Done Done false
8615 2816 Acked Read 9 Sent Done Done false
8616 2817 Acked Write 1 Sent Done Done false
8617 2818 Acked Write 2 Sent Done Done false
8618 2819 Acked Read 2 Sent Done Done false
8619 2820 Acked Read 7 Sent Done Done false
8620 2821 Acked Read 8 Sent Done Done false
8621 2822 Acked Read 4 Sent Done Done false
8622 2823 Acked Read 10 Sent Done Done false
8623 2824 Acked Read 5 Sent Done Done false
8624 2825 Acked Write 2 Sent Done Done false
8625 2826 Acked Flush 0 Sent Done Done false
8626 2827 Acked Read 2 Sent Done Done false
8627 2828 Acked Write 4 Sent Done Done false
8628 2829 Acked Write 1 Sent Done Done false
8629 2830 Acked Read 3 Sent Done Done false
8630 2831 Acked Read 4 Sent Done Done false
8631 2832 Acked Write 4 Sent Done Done false
8632 2833 Acked Write 1 Sent Done Done false
8633 2834 Acked Write 9 Sent Done Done false
8634 2835 Acked Write 5 Sent Done Done false
8635 2836 Acked Read 2 Sent Done Done false
8636 2837 Acked Write 5 Sent Done Done false
8637 2838 Acked Read 3 Sent Done Done false
8638 2839 Acked Write 7 Sent Done Done false
8639 2840 Acked Write 9 Sent Done Done false
8640 2841 Acked Read 5 Sent Done Done false
8641 2842 Acked Read 3 Sent Done Done false
8642 2843 Acked Read 9 Sent Done Done false
8643 2844 Acked Write 9 Sent Done Done false
8644 2845 Acked Write 2 Sent Done Done false
8645 2846 Acked Write 10 Sent Done Done false
8646 2847 Acked Write 8 Sent Done Done false
8647 2848 Acked Read 1 Sent Done Done false
8648 2849 Acked Read 8 Sent Done Done false
8649 2850 Acked Write 10 Sent Done Done false
8650 2851 Acked Write 2 Sent Done Done false
8651 2852 Acked Read 1 Sent Done Done false
8652 2853 Acked Read 4 Sent Done Done false
8653 2854 Acked Read 4 Sent Done Done false
8654 2855 Acked Flush 0 Sent Done Done false
8655 2856 Acked Write 8 Sent Done Done false
8656 2857 Acked Flush 0 Sent Done Done false
8657 2858 Acked Read 4 Sent Done Done false
8658 2859 Acked Write 8 Sent Done Done false
8659 2860 Acked Write 1 Sent Done Done false
8660 2861 Acked Write 2 Sent Done Done false
8661 2862 Acked Write 2 Sent Done Done false
8662 2863 Acked Flush 0 Sent Done Done false
8663 2864 Acked Write 10 Sent Done Done false
8664 2865 Acked Read 3 Sent Done Done false
8665 2866 Acked Read 8 Sent Done Done false
8666 2867 Acked Read 2 Sent Done Done false
8667 2868 Acked Read 6 Sent Done Done false
8668 2869 Acked Write 1 Sent Done Done false
8669 2870 Acked Read 5 Sent Done Done false
8670 2871 Acked Read 6 Sent Done Done false
8671 2872 Acked Read 8 Sent Done Done false
8672 2873 Acked Read 9 Sent Done Done false
8673 2874 Acked Write 8 Sent Done Done false
8674 2875 Acked Read 3 Sent Done Done false
8675 2876 Acked Read 6 Sent Done Done false
8676 2877 Acked Write 9 Sent Done Done false
8677 2878 Acked Write 10 Sent Done Done false
8678 2879 Acked Write 9 Sent Done Done false
8679 2880 Acked Read 1 Sent Done Done false
8680 2881 Acked Read 2 Sent Done Done false
8681 2882 Acked Write 7 Sent Done Done false
8682 2883 Acked Write 10 Sent Done Done false
8683 2884 Acked Flush 0 Sent Done Done false
8684 2885 Acked Write 10 Sent Done Done false
8685 2886 Acked Read 7 Sent Done Done false
8686 2887 Acked Read 1 Sent Done Done false
8687 2888 Acked Read 10 Sent Done Done false
8688 2889 Acked Read 6 Sent Done Done false
8689 2890 Acked Read 8 Sent Done Done false
8690 2891 Acked Read 4 Sent Done Done false
8691 2892 Acked Write 9 Sent Done Done false
8692 2893 Acked Flush 0 Sent Done Done false
8693 2894 Acked Write 8 Sent Done Done false
8694 2895 Acked Read 5 Sent Done Done false
8695 2896 Acked Read 4 Sent Done Done false
8696 2897 Acked Write 3 Sent Done Done false
8697 2898 Acked Read 7 Sent Done Done false
8698 2899 Acked Write 5 Sent Done Done false
8699 2900 Acked Read 1 Sent Done Done false
8700 2901 Acked Read 3 Sent Done Done false
8701 2902 Acked Write 5 Sent Done Done false
8702 2903 Acked Flush 0 Sent Done Done false
8703 2904 Acked Write 3 Sent Done Done false
8704 2905 Acked Write 8 Sent Done Done false
8705 2906 Acked Read 1 Sent Done Done false
8706 2907 Acked Flush 0 Sent Done Done false
8707 2908 Acked Write 3 Sent Done Done false
8708 2909 Acked Write 10 Sent Done Done false
8709 2910 Acked Write 5 Sent Done Done false
8710 2911 Acked Write 6 Sent Done Done false
8711 2912 Acked Read 8 Sent Done Done false
8712 2913 Acked Write 4 Sent Done Done false
8713 2914 Acked Read 4 Sent Done Done false
8714 2915 Acked Write 7 Sent Done Done false
8715 2916 Acked Read 1 Sent Done Done false
8716 2917 Acked Write 3 Sent Done Done false
8717 2918 Acked Read 10 Sent Done Done false
8718 2919 Acked Write 9 Sent Done Done false
8719 2920 Acked Write 8 Sent Done Done false
8720 2921 Acked Flush 0 Sent Done Done false
8721 2922 Acked Read 1 Sent Done Done false
8722 2923 Acked Read 7 Sent Done Done false
8723 2924 Acked Write 2 Sent Done Done false
8724 2925 Acked Write 8 Sent Done Done false
8725 2926 Acked Write 2 Sent Done Done false
8726 2927 Acked Read 5 Sent Done Done false
8727 2928 Acked Write 4 Sent Done Done false
8728 2929 Acked Read 2 Sent Done Done false
8729 2930 Acked Write 2 Sent Done Done false
8730 2931 Acked Read 2 Sent Done Done false
8731 2932 Acked Read 6 Sent Done Done false
8732 2933 Acked Read 9 Sent Done Done false
8733 2934 Acked Flush 0 Sent Done Done false
8734 2935 Acked Read 10 Sent Done Done false
8735 2936 Acked Write 8 Sent Done Done false
8736 2937 Acked Write 6 Sent Done Done false
8737 2938 Acked Read 10 Sent Done Done false
8738 2939 Acked Write 2 Sent Done Done false
8739 2940 Acked Read 7 Sent Done Done false
8740 2941 Acked Flush 0 Sent Done Done false
8741 2942 Acked Write 8 Sent Done Done false
8742 2943 Acked Write 7 Sent Done Done false
8743 2944 Acked Write 9 Sent Done Done false
8744 2945 Acked Read 2 Sent Done Done false
8745 2946 Acked Read 6 Sent Done Done false
8746 2947 Acked Read 7 Sent Done Done false
8747 2948 Acked Write 10 Sent Done Done false
8748 2949 Acked Read 6 Sent Done Done false
8749 2950 Acked Write 8 Sent Done Done false
8750 2951 Acked Write 5 Sent Done Done false
8751 2952 Acked Flush 0 Sent Done Done false
8752 2953 Acked Read 5 Sent Done Done false
8753 2954 Acked Read 3 Sent Done Done false
8754 2955 Acked Read 10 Sent Done Done false
8755 2956 Acked Read 9 Sent Done Done false
8756 2957 Acked Read 4 Sent Done Done false
8757 2958 Acked Write 8 Sent Done Done false
8758 2959 Acked Read 6 Sent Done Done false
8759 2960 Acked Read 2 Sent Done Done false
8760 2961 Acked Write 4 Sent Done Done false
8761 2962 Acked Flush 0 Sent Done Done false
8762 2963 Acked Flush 0 Sent Done Done false
8763 2964 Acked Read 9 Sent Done Done false
8764 2965 Acked Write 4 Sent Done Done false
8765 2966 Acked Flush 0 Sent Done Done false
8766 2967 Acked Write 3 Sent Done Done false
8767 2968 Acked Write 6 Sent Done Done false
8768 2969 Acked Read 9 Sent Done Done false
8769 2970 Acked Read 9 Sent Done Done false
8770 2971 Acked Read 2 Sent Done Done false
8771 2972 Acked Read 4 Sent Done Done false
8772 2973 Acked Flush 0 Sent Done Done false
8773 2974 Acked Read 4 Sent Done Done false
8774 2975 Acked Read 8 Sent Done Done false
8775 2976 Acked Write 7 Sent Done Done false
8776 2977 Acked Read 2 Sent Done Done false
8777 2978 Acked Read 4 Sent Done Done false
8778 2979 Acked Write 6 Sent Done Done false
8779 2980 Acked Flush 0 Sent Done Done false
8780 2981 Acked Write 10 Sent Done Done false
8781 2982 Acked Read 8 Sent Done Done false
8782 2983 Acked Write 2 Sent Done Done false
8783 2984 Acked Read 9 Sent Done Done false
8784 2985 Acked Flush 0 Sent Done Done false
8785 2986 Acked Read 8 Sent Done Done false
8786 2987 Acked Read 6 Sent Done Done false
8787 2988 Acked Read 3 Sent Done Done false
8788 2989 Acked Write 1 Sent Done Done false
8789 2990 Acked Flush 0 Sent Done Done false
8790 2991 Acked Read 7 Sent Done Done false
8791 2992 Acked Write 8 Sent Done Done false
8792 2993 Acked Flush 0 Sent Done Done false
8793 2994 Acked Write 2 Sent Done Done false
8794 2995 Acked Read 5 Sent Done Done false
8795 2996 Acked Read 2 Sent Done Done false
8796 2997 Acked Read 7 Sent Done Done false
8797 2998 Acked Read 2 Sent Done Done false
8798 2999 Acked Read 1 Sent Done Done false
8799 3000 Acked Read 1 Sent Done Done false
8800 3001 Acked Read 8 Sent Done Done false
8801 3002 Acked Read 10 Sent Done Done false
8802 3003 Acked Read 7 Sent Done Done false
8803 3004 Acked Read 4 Sent Done Done false
8804 3005 Acked Write 4 Sent Done Done false
8805 3006 Acked Write 4 Sent Done Done false
8806 3007 Acked Read 10 Sent Done Done false
8807 3008 Acked Write 3 Sent Done Done false
8808 3009 Acked Write 7 Sent Done Done false
8809 3010 Acked Read 7 Sent Done Done false
8810 3011 Acked Read 5 Sent Done Done false
8811 3012 Acked Read 1 Sent Done Done false
8812 3013 Acked Write 3 Sent Done Done false
8813 3014 Acked Read 9 Sent Done Done false
8814 3015 Acked Read 10 Sent Done Done false
8815 3016 Acked Read 2 Sent Done Done false
8816 3017 Acked Flush 0 Sent Done Done false
8817 3018 Acked Read 7 Sent Done Done false
8818 3019 Acked Read 7 Sent Done Done false
8819 3020 Acked Write 8 Sent Done Done false
8820 3021 Acked Read 10 Sent Done Done false
8821 3022 Acked Flush 0 Sent Done Done false
8822 3023 Acked Flush 0 Sent Done Done false
8823 3024 Acked Flush 0 Sent Done Done false
8824 3025 Acked Read 4 Sent Done Done false
8825 3026 Acked Read 3 Sent Done Done false
8826 3027 Acked Read 5 Sent Done Done false
8827 3028 Acked Write 10 Sent Done Done false
8828 3029 Acked Flush 0 Sent Done Done false
8829 3030 Acked Write 5 Sent Done Done false
8830 3031 Acked Write 4 Sent Done Done false
8831 3032 Acked Write 5 Sent Done Done false
8832 3033 Acked Read 4 Sent Done Done false
8833 3034 Acked Flush 0 Sent Done Done false
8834 3035 Acked Flush 0 Sent Done Done false
8835 3036 Acked Read 4 Sent Done Done false
8836 3037 Acked Write 4 Sent Done Done false
8837 3038 Acked Flush 0 Sent Done Done false
8838 3039 Acked Read 2 Sent Done Done false
8839 3040 Acked Write 5 Sent Done Done false
8840 3041 Acked Write 10 Sent Done Done false
8841 3042 Acked Read 10 Sent Done Done false
8842 3043 Acked Read 2 Sent Done Done false
8843 3044 Acked Write 6 Sent Done Done false
8844 3045 Acked Read 7 Sent Done Done false
8845 3046 Acked Flush 0 Sent Done Done false
8846 3047 Acked Write 4 Sent Done Done false
8847 3048 Acked Write 6 Sent Done Done false
8848 3049 Acked Write 10 Sent Done Done false
8849 3050 Acked Read 4 Sent Done Done false
8850 3051 Acked Read 1 Sent Done Done false
8851 3052 Acked Write 8 Sent Done Done false
8852 3053 Acked Write 1 Sent Done Done false
8853 3054 Acked Write 1 Sent Done Done false
8854 3055 Acked Write 9 Sent Done Done false
8855 3056 Acked Read 7 Sent Done Done false
8856 3057 Acked Read 3 Sent Done Done false
8857 3058 Acked Read 4 Sent Done Done false
8858 3059 Acked Write 4 Sent Done Done false
8859 3060 Acked Write 1 Sent Done Done false
8860 3061 Acked Read 2 Sent Done Done false
8861 3062 Acked Read 3 Sent Done Done false
8862 3063 Acked Flush 0 Sent Done Done false
8863 3064 Acked Write 7 Sent Done Done false
8864 3065 Acked Read 1 Sent Done Done false
8865 3066 Acked Read 2 Sent Done Done false
8866 3067 Acked Read 1 Sent Done Done false
8867 3068 Acked Flush 0 Sent Done Done false
8868 3069 Acked Write 4 Sent Done Done false
8869 3070 Acked Read 9 Sent Done Done false
8870 3071 Acked Read 9 Sent Done Done false
8871 3072 Acked Flush 0 Sent Done Done false
8872 3073 Acked Write 8 Sent Done Done false
8873 3074 Acked Read 3 Sent Done Done false
8874 3075 Acked Write 5 Sent Done Done false
8875 3076 Acked Write 8 Sent Done Done false
8876 3077 Acked Write 8 Sent Done Done false
8877 3078 Acked Read 5 Sent Done Done false
8878 3079 Acked Read 7 Sent Done Done false
8879 3080 Acked Read 3 Sent Done Done false
8880 3081 Acked Write 10 Sent Done Done false
8881 3082 Acked Read 8 Sent Done Done false
8882 3083 Acked Write 10 Sent Done Done false
8883 3084 Acked Write 9 Sent Done Done false
8884 3085 Acked Read 2 Sent Done Done false
8885 3086 Acked Write 3 Sent Done Done false
8886 3087 Acked Read 5 Sent Done Done false
8887 3088 Acked Write 1 Sent Done Done false
8888 3089 Acked Read 8 Sent Done Done false
8889 3090 Acked Read 7 Sent Done Done false
8890 3091 Acked Flush 0 Sent Done Done false
8891 3092 Acked Read 9 Sent Done Done false
8892 3093 Acked Read 2 Sent Done Done false
8893 3094 Acked Read 1 Sent Done Done false
8894 3095 Acked Flush 0 Sent Done Done false
8895 3096 Acked Read 3 Sent Done Done false
8896 3097 Acked Read 1 Sent Done Done false
8897 3098 Acked Read 6 Sent Done Done false
8898 3099 Acked Write 9 Sent Done Done false
8899 3100 Acked Read 8 Sent Done Done false
8900 3101 Acked Read 3 Sent Done Done false
8901 3102 Acked Read 8 Sent Done Done false
8902 3103 Acked Write 7 Sent Done Done false
8903 3104 Acked Read 8 Sent Done Done false
8904 3105 Acked Read 9 Sent Done Done false
8905 3106 Acked Read 8 Sent Done Done false
8906 3107 Acked Read 2 Sent Done Done false
8907 3108 Acked Write 7 Sent Done Done false
8908 3109 Acked Write 9 Sent Done Done false
8909 3110 Acked Write 5 Sent Done Done false
8910 3111 Acked Read 4 Sent Done Done false
8911 3112 Acked Read 1 Sent Done Done false
8912 3113 Acked Write 2 Sent Done Done false
8913 3114 Acked Flush 0 Sent Done Done false
8914 3115 Acked Read 9 Sent Done Done false
8915 3116 Acked Flush 0 Sent Done Done false
8916 STATES DS:0 DS:1 DS:2 TOTAL
8917 Sent 302 0 0 302
8918 Done 0 302 302 604
8919 Skipped 0 0 0 0
8920 Error 0 0 0 0
8921 Last Flush: 2814 3116 3116
8922 Downstairs last five completed: 2814 2813 2812 2811 2810
8923 Upstairs last five completed: 3116 3115 3114 3113 3112
8924 CLIENT: Up:0 ds:302 act:2
89252024-11-07T14:05:09.583ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89262024-11-07T14:05:09.583ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:56316) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89272024-11-07T14:05:09.584ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89282024-11-07T14:05:09.584ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89292024-11-07T14:05:09.584ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89302024-11-07T14:05:09.584ZINFOcrucible: send last flush ID to this DS: 2814 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89312024-11-07T14:05:09.585ZINFOcrucible: Replied this last flush ID: 2814 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89322024-11-07T14:05:09.585ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89332024-11-07T14:05:09.585ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89342024-11-07T14:05:09.585ZINFOcrucible: [0] client re-new 302 jobs since flush 2814 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89352024-11-07T14:05:09.585ZINFOcrucible: [0] Replayed 302 jobs since flush: 2814 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
8936 ----------------------------------------------------------------
8937 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8938 Downstairs last five completed: 3116 3115 3114 3113 3112
8939 Upstairs last five completed: 3116 3115 3114 3113 3112
8940 CLIENT: Up:0 ds:0 act:3
8941 Replay: All jobs finished
8942 Main task has work to do, go find it
8943 got dsc Stop(0)
8944 stop 0
8945 Nov 07 14:05:11.876 INFO request completed, latency_us: 367, response_code: 204, uri: /stop/cid/0, method: POST, req_id: d46880a3-7231-4b63-b5e2-56dc7bcb56c5, remote_addr: 127.[8810] Got stop action so:false kr:false
8946 0.0.1:34100, local_addr: 127.0.0.1:9998
8947 Nov 07 14:05:11.876 INFO request completed, latency_us: 136, response_code: 200, uri: /state/cid/0, method: GET, req_id: c68bf97e-7413-4e90-a168-4b4c5defd2af, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
89482024-11-07T14:05:11.876ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89492024-11-07T14:05:11.876ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89502024-11-07T14:05:11.876ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89512024-11-07T14:05:11.877ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89522024-11-07T14:05:11.877ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
89532024-11-07T14:05:11.877ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
8954 [8810] Exited with: ExitStatus(unix_wait_status(9))
8955 [8810][0] reports Exit
8956 Nov 07 14:05:15.879 INFO request completed, latency_us: 209, response_code: 200, uri: /state/cid/0, method: GET, req_id: c0b09c87-1ebb-45e6-92cf-56697216cae8, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
8957 001/300 Read block 1517 len 40960
8958 002/300 Write block 3115 len 40960 data: 2 2 2 2 2 2 2 2 2 3
8959 003/300 Write block 3269 len 40960 data: 3 3 3 4 4 3 3 2 2 2
8960 004/300 Write block 1728 len 8192 data: 2 2
8961 005/300 Flush
8962 006/300 Flush
8963 007/300 Write block 147 len 32768 data: 3 2 2 2 3 3 3 4
8964 008/300 Read block 4276 len 20480
8965 009/300 Write block 3178 len 4096 data: 2
8966 010/300 Flush
8967 011/300 Read block 1593 len 40960
8968 012/300 Read block 2693 len 24576
8969 013/300 Read block 1236 len 20480
8970 014/300 Flush
8971 015/300 Flush
8972 016/300 Read block 3589 len 16384
8973 017/300 Write block 3495 len 4096 data: 2
8974 018/300 Read block 117 len 24576
8975 019/300 Flush
8976 020/300 Write block 1836 len 8192 data: 2 3
8977 021/300 Flush
8978 022/300 Read block 2175 len 12288
8979 023/300 Flush
8980 024/300 Read block 2129 len 28672
8981 025/300 Read block 207 len 32768
8982 026/300 Write block 376 len 32768 data: 2 2 2 2 3 4 4 4
8983 027/300 Read block 3784 len 8192
8984 028/300 Read block 1238 len 8192
8985 029/300 Read block 230 len 36864
8986 030/300 Write block 1473 len 36864 data: 2 2 2 2 2 2 2 2 2
8987 031/300 Read block 2549 len 32768
8988 032/300 Read block 922 len 20480
8989 033/300 Read block 2922 len 36864
8990 034/300 Read block 884 len 28672
8991 035/300 Write block 197 len 8192 data: 3 3
8992 036/300 Read block 3311 len 20480
8993 037/300 Write block 4148 len 16384 data: 3 3 3 3
8994 038/300 Read block 657 len 8192
8995 039/300 Flush
8996 040/300 Read block 3130 len 28672
8997 041/300 Read block 2354 len 24576
8998 042/300 Read block 3409 len 40960
8999 043/300 Flush
9000 044/300 Read block 3465 len 36864
9001 045/300 Read block 533 len 8192
9002 046/300 Write block 2463 len 12288 data: 2 2 2
9003 047/300 Read block 1107 len 40960
9004 048/300 Flush
9005 049/300 Flush
9006 050/300 Flush
9007 051/300 Write block 1560 len 36864 data: 4 4 3 3 4 3 3 3 3
9008 052/300 Read block 721 len 28672
9009 053/300 Read block 4972 len 12288
9010 054/300 Read block 3241 len 32768
9011 055/300 Read block 2755 len 36864
9012 056/300 Flush
9013 057/300 Read block 4560 len 40960
9014 058/300 Write block 1088 len 8192 data: 3 5
9015 059/300 Read block 1652 len 32768
9016 060/300 Write block 3325 len 8192 data: 3 3
9017 061/300 Read block 3076 len 28672
9018 062/300 Read block 1997 len 16384
9019 063/300 Flush
9020 064/300 Write block 2190 len 16384 data: 2 2 2 3
9021 065/300 Read block 2530 len 8192
9022 066/300 Flush
9023 067/300 Read block 4892 len 20480
9024 068/300 Read block 2639 len 8192
9025 069/300 Read block 3841 len 8192
9026 070/300 Read block 4715 len 16384
9027 071/300 Write block 903 len 8192 data: 5 5
9028 072/300 Read block 2075 len 28672
9029 073/300 Write block 3859 len 20480 data: 2 2 3 3 3
9030 074/300 Flush
9031 075/300 Write block 368 len 24576 data: 2 2 2 2 2 2
9032 076/300 Read block 3327 len 24576
9033 077/300 Read block 1179 len 36864
9034 078/300 Write block 2508 len 20480 data: 2 2 2 2 2
9035 079/300 Write block 3071 len 24576 data: 2 2 2 2 2 2
9036 080/300 Read block 3983 len 36864
9037 081/300 Write block 1082 len 32768 data: 5 5 3 3 3 3 4 6
9038 082/300 Read block 4348 len 28672
9039 083/300 Read block 1551 len 40960
9040 084/300 Read block 3917 len 20480
9041 085/300 Write block 2786 len 36864 data: 3 2 2 2 2 2 2 2 2
9042 086/300 Write block 1661 len 40960 data: 3 3 3 3 3 4 6 4 4 4
9043 087/300 Read block 112 len 8192
9044 088/300 Write block 4171 len 16384 data: 2 2 3 3
9045 089/300 Read block 2057 len 16384
9046 090/300 Write block 2144 len 4096 data: 3
9047 091/300 Read block 3153 len 40960
9048 092/300 Flush
9049 093/300 Read block 4290 len 32768
9050 094/300 Write block 3058 len 28672 data: 3 3 3 3 3 3 3
9051 095/300 Read block 3988 len 24576
9052 096/300 Write block 226 len 8192 data: 2 2
9053 097/300 Read block 2551 len 4096
9054 098/300 Write block 4220 len 40960 data: 2 2 2 2 2 2 2 3 3 3
9055 099/300 Read block 3607 len 4096
9056 100/300 Write block 933 len 32768 data: 2 2 2 2 2 2 3 3
9057 101/300 Write block 4697 len 24576 data: 4 4 3 3 4 3
9058 102/300 Read block 2554 len 28672
9059 103/300 Write block 622 len 8192 data: 2 2
9060 104/300 Read block 4393 len 16384
9061 105/300 Write block 3300 len 12288 data: 2 2 2
9062 106/300 Read block 3513 len 36864
9063 107/300 Write block 2427 len 16384 data: 4 4 3 3
9064 108/300 Write block 1978 len 28672 data: 2 2 2 3 3 3 4
9065 109/300 Write block 3765 len 8192 data: 2 2
9066 110/300 Write block 1843 len 40960 data: 4 2 2 2 3 3 3 5 4 5
9067 111/300 Flush
9068 112/300 Flush
9069 113/300 Read block 1657 len 40960
9070 114/300 Flush
9071 115/300 Read block 3549 len 20480
9072 116/300 Read block 2069 len 36864
9073 117/300 Write block 4788 len 20480 data: 2 2 2 2 2
9074 118/300 Read block 280 len 8192
9075 119/300 Flush
9076 120/300 Write block 3548 len 32768 data: 2 2 2 2 3 3 2 2
9077 121/300 Read block 3741 len 36864
9078 122/300 Read block 1373 len 16384
9079 123/300 Read block 948 len 36864
9080 124/300 Read block 4081 len 20480
9081 125/300 Flush
9082 126/300 Write block 4085 len 36864 data: 2 2 2 2 2 2 2 2 2
9083 127/300 Flush
9084 128/300 Write block 706 len 36864 data: 5 5 4 3 3 2 2 2 2
9085 129/300 Write block 159 len 36864 data: 4 5 4 3 3 3 3 3 3
9086 130/300 Write block 130 len 36864 data: 3 2 2 2 2 2 2 2 2
9087 131/300 Flush
9088 132/300 Write block 3691 len 32768 data: 2 3 3 3 3 2 2 3
9089 133/300 Read block 519 len 12288
9090 134/300 Read block 4156 len 24576
9091 135/300 Read block 3347 len 4096
9092 136/300 Flush
9093 137/300 Read block 1493 len 4096
9094 138/300 Read block 722 len 16384
9095 139/300 Read block 2828 len 16384
9096 140/300 Flush
9097 141/300 Read block 4651 len 40960
9098 142/300 Write block 4196 len 36864 data: 2 2 3 3 3 3 3 3 3
9099 143/300 Write block 2936 len 8192 data: 2 2
9100 144/300 Read block 1930 len 36864
9101 145/300 Read block 3742 len 16384
9102 146/300 Read block 4930 len 28672
9103 147/300 Write block 1844 len 12288 data: 3 3 3
9104 148/300 Write block 3358 len 32768 data: 3 2 2 2 2 2 2 2
9105 149/300 Flush
9106 150/300 Write block 157 len 28672 data: 4 4 5 6 5 4 4
9107 151/300 Read block 3154 len 16384
9108 152/300 Read block 4932 len 8192
9109 153/300 Read block 69 len 8192
9110 154/300 Write block 3883 len 12288 data: 5 4 2
9111 155/300 Flush
9112 156/300 Write block 4796 len 32768 data: 2 2 2 3 3 3 4 4
9113 157/300 Flush
9114 158/300 Flush
9115 159/300 Read block 4748 len 40960
9116 160/300 Read block 3275 len 24576
9117 161/300 Write block 2301 len 20480 data: 4 4 4 5 6
9118 162/300 Write block 4502 len 12288 data: 3 3 3
9119 163/300 Read block 3240 len 12288
9120 164/300 Read block 3439 len 28672
9121 165/300 Read block 42 len 12288
9122 166/300 Read block 2287 len 12288
9123 167/300 Read block 4868 len 28672
9124 168/300 Read block 3784 len 36864
9125 169/300 Write block 3711 len 36864 data: 3 3 2 2 2 2 3 3 3
9126 170/300 Read block 3596 len 40960
9127 171/300 Read block 1538 len 36864
9128 172/300 Read block 2058 len 4096
9129 173/300 Read block 3596 len 20480
9130 174/300 Read block 2495 len 32768
9131 175/300 Read block 3800 len 24576
9132 176/300 Write block 823 len 16384 data: 4 4 4 3
9133 177/300 Read block 3386 len 20480
9134 178/300 Flush
9135 179/300 Write block 1353 len 8192 data: 2 4
9136 180/300 Read block 4882 len 28672
9137 181/300 Write block 2303 len 28672 data: 5 6 7 5 4 4 3
9138 182/300 Write block 750 len 28672 data: 3 4 4 3 3 2 2
9139 183/300 Write block 1117 len 16384 data: 2 2 2 2
9140 184/300 Write block 3695 len 40960 data: 4 3 3 4 3 3 3 3 4 4
9141 185/300 Write block 3533 len 28672 data: 3 3 3 2 2 2 2
9142 186/300 Read block 4646 len 36864
9143 187/300 Write block 3795 len 40960 data: 5 4 4 4 4 4 3 3 3 3
9144 188/300 Read block 4571 len 32768
9145 189/300 Read block 2687 len 40960
9146 190/300 Write block 4575 len 8192 data: 4 4
9147 191/300 Read block 4952 len 16384
9148 192/300 Write block 4362 len 40960 data: 3 2 2 2 3 4 4 4 4 4
9149 193/300 Read block 3007 len 16384
9150 194/300 Read block 1958 len 40960
9151 195/300 Read block 1235 len 12288
9152 196/300 Write block 3292 len 28672 data: 3 2 2 2 2 2 2
9153 197/300 Write block 1154 len 12288 data: 3 3 3
9154 198/300 Write block 3660 len 40960 data: 4 4 3 3 3 3 3 3 2 3
9155 199/300 Write block 3176 len 16384 data: 2 2 3 2
9156 200/300 Write block 149 len 4096 data: 3
9157 201/300 Write block 3283 len 16384 data: 3 2 3 3
9158 202/300 Read block 3639 len 36864
9159 203/300 Flush
9160 204/300 Write block 651 len 24576 data: 5 4 4 3 3 3
9161 205/300 Read block 1197 len 32768
9162 206/300 Read block 3968 len 20480
9163 207/300 Write block 4348 len 40960 data: 4 4 4 4 4 3 2 3 3 3
9164 208/300 Flush
9165 209/300 Read block 2520 len 28672
9166 210/300 Write block 3507 len 20480 data: 3 3 4 4 4
9167 211/300 Read block 1142 len 4096
9168 212/300 Write block 1449 len 12288 data: 6 6 3
9169 213/300 Write block 1864 len 4096 data: 2
9170 214/300 Flush
9171 215/300 Read block 1553 len 28672
9172 216/300 Write block 1095 len 28672 data: 2 2 2 2 2 3 3
9173 217/300 Flush
9174 218/300 Read block 2525 len 36864
9175 219/300 Flush
9176 220/300 Write block 1061 len 8192 data: 3 3
9177 221/300 Read block 4668 len 8192
9178 222/300 Write block 3011 len 40960 data: 4 4 3 3 3 3 3 2 2 2
9179 223/300 Read block 4578 len 36864
9180 224/300 Read block 3612 len 40960
9181 225/300 Write block 2656 len 16384 data: 2 2 3 3
9182 226/300 Read block 2504 len 12288
9183 227/300 Read block 1827 len 32768
9184 228/300 Write block 1522 len 16384 data: 3 2 2 3
9185 229/300 Read block 1691 len 32768
9186 230/300 Read block 3077 len 12288
9187 231/300 Read block 3619 len 8192
9188 232/300 Read block 1091 len 16384
9189 233/300 Read block 4640 len 20480
9190 234/300 Read block 3167 len 24576
9191 235/300 Read block 382 len 24576
9192 236/300 Write block 919 len 8192 data: 4 2
9193 237/300 Write block 1222 len 36864 data: 2 2 2 2 2 2 2 2 3
9194 238/300 Write block 3302 len 8192 data: 3 2
9195 239/300 Read block 3193 len 40960
9196 240/300 Write block 3279 len 8192 data: 2 2
9197 241/300 Read block 4440 len 32768
9198 242/300 Write block 2006 len 32768 data: 3 4 4 4 4 4 5 6
9199 243/300 Write block 2008 len 4096 data: 5
9200 244/300 Read block 4885 len 24576
9201 245/300 Write block 3937 len 8192 data: 2 2
9202 246/300 Read block 2658 len 4096
9203 247/300 Flush
9204 248/300 Write block 3567 len 12288 data: 3 3 3
9205 249/300 Read block 659 len 24576
9206 250/300 Write block 3817 len 4096 data: 4
9207 251/300 Write block 304 len 24576 data: 3 4 3 4 4 3
9208 252/300 Write block 2236 len 24576 data: 2 2 2 2 2 2
9209 253/300 Read block 2874 len 4096
9210 254/300 Read block 2005 len 36864
9211 255/300 Read block 2996 len 32768
9212 256/300 Read block 3410 len 36864
9213 257/300 Flush
9214 258/300 Write block 345 len 24576 data: 4 3 3 3 3 3
9215 259/300 Write block 186 len 20480 data: 4 3 3 4 4
9216 260/300 Flush
9217 261/300 Write block 3608 len 16384 data: 3 3 3 3
9218 262/300 Read block 4538 len 8192
9219 263/300 Write block 380 len 32768 data: 4 5 5 5 3 2 2 2
9220 264/300 Write block 2720 len 16384 data: 3 4 4 4
9221 265/300 Read block 227 len 8192
9222 266/300 Flush
9223 267/300 Flush
9224 268/300 Read block 3759 len 28672
9225 269/300 Write block 2419 len 12288 data: 3 3 4
9226 270/300 Write block 4740 len 16384 data: 4 5 4 3
9227 271/300 Flush
9228 272/300 Read block 1686 len 32768
9229 273/300 Write block 743 len 16384 data: 3 3 4 3
9230 274/300 Write block 1563 len 24576 data: 4 5 4 4 4 4
9231 275/300 Read block 2137 len 32768
9232 276/300 Read block 2451 len 28672
9233 277/300 Write block 527 len 4096 data: 4
9234 278/300 Write block 4337 len 16384 data: 5 5 4 4
9235 279/300 Write block 4112 len 16384 data: 3 3 2 3
9236 280/300 Write block 983 len 8192 data: 3 3
9237 281/300 Read block 4544 len 32768
9238 282/300 Write block 3815 len 20480 data: 5 5 5 4 4
9239 283/300 Read block 1450 len 20480
9240 284/300 Read block 4616 len 40960
9241 285/300 Flush
9242 286/300 Write block 547 len 4096 data: 4
9243 287/300 Read block 3136 len 36864
9244 288/300 Read block 1180 len 8192
9245 289/300 Read block 999 len 4096
9246 290/300 Read block 4928 len 12288
9247 291/300 Read block 4724 len 32768
9248 292/300 Write block 3580 len 28672 data: 2 3 3 3 3 3 3
9249 293/300 Write block 1455 len 24576 data: 2 2 2 2 2 2
9250 294/300 Read block 4371 len 24576
9251 295/300 Read block 3804 len 28672
9252 296/300 Read block 350 len 20480
9253 297/300 Write block 4477 len 36864 data: 4 4 4 4 4 4 3 3 3
9254 298/300 Read block 2594 len 32768
9255 299/300 Read block 81 len 4096
9256 300/300 Read block 2865 len 28672
9257 301/300 Read block 3741 len 28672
9258 Main task has work to do, go find it
9259 got dsc Start(0)
9260 start 0
9261 Nov 07 14:05:16.180 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/0, method: POST, req_id: 8257c1e9-c4bb-457e-9bd8-1a554596bd4f, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
9262 Replay: started 0, returned:Ok(())
9263 ----------------------------------------------------------------
9264 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9265 States: Offline Active Active
9266 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9267 3117 Acked Read 10 Sent Done Done false
9268 3118 Acked Write 10 Sent Done Done false
9269 3119 Acked Write 10 Sent Done Done false
9270 3120 Acked Write 2 Sent Done Done false
9271 3121 Acked Flush 0 Sent Done Done false
9272 3122 Acked Flush 0 Sent Done Done false
9273 3123 Acked Write 8 Sent Done Done false
9274 3124 Acked Read 5 Sent Done Done false
9275 3125 Acked Write 1 Sent Done Done false
9276 3126 Acked Flush 0 Sent Done Done false
9277 3127 Acked Read 10 Sent Done Done false
9278 3128 Acked Read 6 Sent Done Done false
9279 3129 Acked Read 5 Sent Done Done false
9280 3130 Acked Flush 0 Sent Done Done false
9281 3131 Acked Flush 0 Sent Done Done false
9282 3132 Acked Read 4 Sent Done Done false
9283 3133 Acked Write 1 Sent Done Done false
9284 3134 Acked Read 6 Sent Done Done false
9285 3135 Acked Flush 0 Sent Done Done false
9286 3136 Acked Write 2 Sent Done Done false
9287 3137 Acked Flush 0 Sent Done Done false
9288 3138 Acked Read 3 Sent Done Done false
9289 3139 Acked Flush 0 Sent Done Done false
9290 3140 Acked Read 7 Sent Done Done false
9291 3141 Acked Read 8 Sent Done Done false
9292 3142 Acked Write 8 Sent Done Done false
9293 3143 Acked Read 2 Sent Done Done false
9294 3144 Acked Read 2 Sent Done Done false
9295 3145 Acked Read 9 Sent Done Done false
9296 3146 Acked Write 9 Sent Done Done false
9297 3147 Acked Read 8 Sent Done Done false
9298 3148 Acked Read 5 Sent Done Done false
9299 3149 Acked Read 9 Sent Done Done false
9300 3150 Acked Read 7 Sent Done Done false
9301 3151 Acked Write 2 Sent Done Done false
9302 3152 Acked Read 5 Sent Done Done false
9303 3153 Acked Write 4 Sent Done Done false
9304 3154 Acked Read 2 Sent Done Done false
9305 3155 Acked Flush 0 Sent Done Done false
9306 3156 Acked Read 7 Sent Done Done false
9307 3157 Acked Read 6 Sent Done Done false
9308 3158 Acked Read 10 Sent Done Done false
9309 3159 Acked Flush 0 Sent Done Done false
9310 3160 Acked Read 9 Sent Done Done false
9311 3161 Acked Read 2 Sent Done Done false
9312 3162 Acked Write 3 Sent Done Done false
9313 3163 Acked Read 10 Sent Done Done false
9314 3164 Acked Flush 0 Sent Done Done false
9315 3165 Acked Flush 0 Sent Done Done false
9316 3166 Acked Flush 0 Sent Done Done false
9317 3167 Acked Write 9 Sent Done Done false
9318 3168 Acked Read 7 Sent Done Done false
9319 3169 Acked Read 3 Sent Done Done false
9320 3170 Acked Read 8 Sent Done Done false
9321 3171 Acked Read 9 Sent Done Done false
9322 3172 Acked Flush 0 Sent Done Done false
9323 3173 Acked Read 10 Sent Done Done false
9324 3174 Acked Write 2 Sent Done Done false
9325 3175 Acked Read 8 Sent Done Done false
9326 3176 Acked Write 2 Sent Done Done false
9327 3177 Acked Read 7 Sent Done Done false
9328 3178 Acked Read 4 Sent Done Done false
9329 3179 Acked Flush 0 Sent Done Done false
9330 3180 Acked Write 4 Sent Done Done false
9331 3181 Acked Read 2 Sent Done Done false
9332 3182 Acked Flush 0 Sent Done Done false
9333 3183 Acked Read 5 Sent Done Done false
9334 3184 Acked Read 2 Sent Done Done false
9335 3185 Acked Read 2 Sent Done Done false
9336 3186 Acked Read 4 Sent Done Done false
9337 3187 Acked Write 2 Sent Done Done false
9338 3188 Acked Read 7 Sent Done Done false
9339 3189 Acked Write 5 Sent Done Done false
9340 3190 Acked Flush 0 Sent Done Done false
9341 3191 Acked Write 6 Sent Done Done false
9342 3192 Acked Read 6 Sent Done Done false
9343 3193 Acked Read 9 Sent Done Done false
9344 3194 Acked Write 5 Sent Done Done false
9345 3195 Acked Write 6 Sent Done Done false
9346 3196 Acked Read 9 Sent Done Done false
9347 3197 Acked Write 8 Sent Done Done false
9348 3198 Acked Read 7 Sent Done Done false
9349 3199 Acked Read 10 Sent Done Done false
9350 3200 Acked Read 5 Sent Done Done false
9351 3201 Acked Write 9 Sent Done Done false
9352 3202 Acked Flush 0 Sent Done Done false
9353 3203 Acked Write 10 Sent Done Done false
9354 3204 Acked Read 2 Sent Done Done false
9355 3205 Acked Write 4 Sent Done Done false
9356 3206 Acked Read 4 Sent Done Done false
9357 3207 Acked Write 1 Sent Done Done false
9358 3208 Acked Read 10 Sent Done Done false
9359 3209 Acked Flush 0 Sent Done Done false
9360 3210 Acked Read 8 Sent Done Done false
9361 3211 Acked Write 7 Sent Done Done false
9362 3212 Acked Read 6 Sent Done Done false
9363 3213 Acked Write 2 Sent Done Done false
9364 3214 Acked Read 1 Sent Done Done false
9365 3215 Acked Write 10 Sent Done Done false
9366 3216 Acked Read 1 Sent Done Done false
9367 3217 Acked Write 8 Sent Done Done false
9368 3218 Acked Write 6 Sent Done Done false
9369 3219 Acked Read 7 Sent Done Done false
9370 3220 Acked Write 2 Sent Done Done false
9371 3221 Acked Read 4 Sent Done Done false
9372 3222 Acked Write 3 Sent Done Done false
9373 3223 Acked Read 9 Sent Done Done false
9374 3224 Acked Write 4 Sent Done Done false
9375 3225 Acked Write 7 Sent Done Done false
9376 3226 Acked Write 2 Sent Done Done false
9377 3227 Acked Write 10 Sent Done Done false
9378 3228 Acked Flush 0 Sent Done Done false
9379 3229 Acked Flush 0 Sent Done Done false
9380 3230 Acked Read 10 Sent Done Done false
9381 3231 Acked Flush 0 Sent Done Done false
9382 3232 Acked Read 5 Sent Done Done false
9383 3233 Acked Read 9 Sent Done Done false
9384 3234 Acked Write 5 Sent Done Done false
9385 3235 Acked Read 2 Sent Done Done false
9386 3236 Acked Flush 0 Sent Done Done false
9387 3237 Acked Write 8 Sent Done Done false
9388 3238 Acked Read 9 Sent Done Done false
9389 3239 Acked Read 4 Sent Done Done false
9390 3240 Acked Read 9 Sent Done Done false
9391 3241 Acked Read 5 Sent Done Done false
9392 3242 Acked Flush 0 Sent Done Done false
9393 3243 Acked Write 9 Sent Done Done false
9394 3244 Acked Flush 0 Sent Done Done false
9395 3245 Acked Write 9 Sent Done Done false
9396 3246 Acked Write 9 Sent Done Done false
9397 3247 Acked Write 9 Sent Done Done false
9398 3248 Acked Flush 0 Sent Done Done false
9399 3249 Acked Write 8 Sent Done Done false
9400 3250 Acked Read 3 Sent Done Done false
9401 3251 Acked Read 6 Sent Done Done false
9402 3252 Acked Read 1 Sent Done Done false
9403 3253 Acked Flush 0 Sent Done Done false
9404 3254 Acked Read 1 Sent Done Done false
9405 3255 Acked Read 4 Sent Done Done false
9406 3256 Acked Read 4 Sent Done Done false
9407 3257 Acked Flush 0 Sent Done Done false
9408 3258 Acked Read 10 Sent Done Done false
9409 3259 Acked Write 9 Sent Done Done false
9410 3260 Acked Write 2 Sent Done Done false
9411 3261 Acked Read 9 Sent Done Done false
9412 3262 Acked Read 4 Sent Done Done false
9413 3263 Acked Read 7 Sent Done Done false
9414 3264 Acked Write 3 Sent Done Done false
9415 3265 Acked Write 8 Sent Done Done false
9416 3266 Acked Flush 0 Sent Done Done false
9417 3267 Acked Write 7 Sent Done Done false
9418 3268 Acked Read 4 Sent Done Done false
9419 3269 Acked Read 2 Sent Done Done false
9420 3270 Acked Read 2 Sent Done Done false
9421 3271 Acked Write 3 Sent Done Done false
9422 3272 Acked Flush 0 Sent Done Done false
9423 3273 Acked Write 8 Sent Done Done false
9424 3274 Acked Flush 0 Sent Done Done false
9425 3275 Acked Flush 0 Sent Done Done false
9426 3276 Acked Read 10 Sent Done Done false
9427 3277 Acked Read 6 Sent Done Done false
9428 3278 Acked Write 5 Sent Done Done false
9429 3279 Acked Write 3 Sent Done Done false
9430 3280 Acked Read 3 Sent Done Done false
9431 3281 Acked Read 7 Sent Done Done false
9432 3282 Acked Read 3 Sent Done Done false
9433 3283 Acked Read 3 Sent Done Done false
9434 3284 Acked Read 7 Sent Done Done false
9435 3285 Acked Read 9 Sent Done Done false
9436 3286 Acked Write 9 Sent Done Done false
9437 3287 Acked Read 10 Sent Done Done false
9438 3288 Acked Read 9 Sent Done Done false
9439 3289 Acked Read 1 Sent Done Done false
9440 3290 Acked Read 5 Sent Done Done false
9441 3291 Acked Read 8 Sent Done Done false
9442 3292 Acked Read 6 Sent Done Done false
9443 3293 Acked Write 4 Sent Done Done false
9444 3294 Acked Read 5 Sent Done Done false
9445 3295 Acked Flush 0 Sent Done Done false
9446 3296 Acked Write 2 Sent Done Done false
9447 3297 Acked Read 7 Sent Done Done false
9448 3298 Acked Write 7 Sent Done Done false
9449 3299 Acked Write 7 Sent Done Done false
9450 3300 Acked Write 4 Sent Done Done false
9451 3301 Acked Write 10 Sent Done Done false
9452 3302 Acked Write 7 Sent Done Done false
9453 3303 Acked Read 9 Sent Done Done false
9454 3304 Acked Write 10 Sent Done Done false
9455 3305 Acked Read 8 Sent Done Done false
9456 3306 Acked Read 10 Sent Done Done false
9457 3307 Acked Write 2 Sent Done Done false
9458 3308 Acked Read 4 Sent Done Done false
9459 3309 Acked Write 10 Sent Done Done false
9460 3310 Acked Read 4 Sent Done Done false
9461 3311 Acked Read 10 Sent Done Done false
9462 3312 Acked Read 3 Sent Done Done false
9463 3313 Acked Write 7 Sent Done Done false
9464 3314 Acked Write 3 Sent Done Done false
9465 3315 Acked Write 10 Sent Done Done false
9466 3316 Acked Write 4 Sent Done Done false
9467 3317 Acked Write 1 Sent Done Done false
9468 3318 Acked Write 4 Sent Done Done false
9469 3319 Acked Read 9 Sent Done Done false
9470 3320 Acked Flush 0 Sent Done Done false
9471 3321 Acked Write 6 Sent Done Done false
9472 3322 Acked Read 8 Sent Done Done false
9473 3323 Acked Read 5 Sent Done Done false
9474 3324 Acked Write 10 Sent Done Done false
9475 3325 Acked Flush 0 Sent Done Done false
9476 3326 Acked Read 7 Sent Done Done false
9477 3327 Acked Write 5 Sent Done Done false
9478 3328 Acked Read 1 Sent Done Done false
9479 3329 Acked Write 3 Sent Done Done false
9480 3330 Acked Write 1 Sent Done Done false
9481 3331 Acked Flush 0 Sent Done Done false
9482 3332 Acked Read 7 Sent Done Done false
9483 3333 Acked Write 7 Sent Done Done false
9484 3334 Acked Flush 0 Sent Done Done false
9485 3335 Acked Read 9 Sent Done Done false
9486 3336 Acked Flush 0 Sent Done Done false
9487 3337 Acked Write 2 Sent Done Done false
9488 3338 Acked Read 2 Sent Done Done false
9489 3339 Acked Write 10 Sent Done Done false
9490 3340 Acked Read 9 Sent Done Done false
9491 3341 Acked Read 10 Sent Done Done false
9492 3342 Acked Write 4 Sent Done Done false
9493 3343 Acked Read 3 Sent Done Done false
9494 3344 Acked Read 8 Sent Done Done false
9495 3345 Acked Write 4 Sent Done Done false
9496 3346 Acked Read 8 Sent Done Done false
9497 3347 Acked Read 3 Sent Done Done false
9498 3348 Acked Read 2 Sent Done Done false
9499 3349 Acked Read 4 Sent Done Done false
9500 3350 Acked Read 5 Sent Done Done false
9501 3351 Acked Read 6 Sent Done Done false
9502 3352 Acked Read 6 Sent Done Done false
9503 3353 Acked Write 2 Sent Done Done false
9504 3354 Acked Write 9 Sent Done Done false
9505 3355 Acked Write 2 Sent Done Done false
9506 3356 Acked Read 10 Sent Done Done false
9507 3357 Acked Write 2 Sent Done Done false
9508 3358 Acked Read 8 Sent Done Done false
9509 3359 Acked Write 8 Sent Done Done false
9510 3360 Acked Write 1 Sent Done Done false
9511 3361 Acked Read 6 Sent Done Done false
9512 3362 Acked Write 2 Sent Done Done false
9513 3363 Acked Read 1 Sent Done Done false
9514 3364 Acked Flush 0 Sent Done Done false
9515 3365 Acked Write 3 Sent Done Done false
9516 3366 Acked Read 6 Sent Done Done false
9517 3367 Acked Write 1 Sent Done Done false
9518 3368 Acked Write 6 Sent Done Done false
9519 3369 Acked Write 6 Sent Done Done false
9520 3370 Acked Read 1 Sent Done Done false
9521 3371 Acked Read 9 Sent Done Done false
9522 3372 Acked Read 8 Sent Done Done false
9523 3373 Acked Read 9 Sent Done Done false
9524 3374 Acked Flush 0 Sent Done Done false
9525 3375 Acked Write 6 Sent Done Done false
9526 3376 Acked Write 5 Sent Done Done false
9527 3377 Acked Flush 0 Sent Done Done false
9528 3378 Acked Write 4 Sent Done Done false
9529 3379 Acked Read 2 Sent Done Done false
9530 3380 Acked Write 8 Sent Done Done false
9531 3381 Acked Write 4 Sent Done Done false
9532 3382 Acked Read 2 Sent Done Done false
9533 3383 Acked Flush 0 Sent Done Done false
9534 3384 Acked Flush 0 Sent Done Done false
9535 3385 Acked Read 7 Sent Done Done false
9536 3386 Acked Write 3 Sent Done Done false
9537 3387 Acked Write 4 Sent Done Done false
9538 3388 Acked Flush 0 Sent Done Done false
9539 3389 Acked Read 8 Sent Done Done false
9540 3390 Acked Write 4 Sent Done Done false
9541 3391 Acked Write 6 Sent Done Done false
9542 3392 Acked Read 8 Sent Done Done false
9543 3393 Acked Read 7 Sent Done Done false
9544 3394 Acked Write 1 Sent Done Done false
9545 3395 Acked Write 4 Sent Done Done false
9546 3396 Acked Write 4 Sent Done Done false
9547 3397 Acked Write 2 Sent Done Done false
9548 3398 Acked Read 8 Sent Done Done false
9549 3399 Acked Write 5 Sent Done Done false
9550 3400 Acked Read 5 Sent Done Done false
9551 3401 Acked Read 10 Sent Done Done false
9552 3402 Acked Flush 0 Sent Done Done false
9553 3403 Acked Write 1 Sent Done Done false
9554 3404 Acked Read 9 Sent Done Done false
9555 3405 Acked Read 2 Sent Done Done false
9556 3406 Acked Read 1 Sent Done Done false
9557 3407 Acked Read 3 Sent Done Done false
9558 3408 Acked Read 8 Sent Done Done false
9559 3409 Acked Write 7 Sent Done Done false
9560 3410 Acked Write 6 Sent Done Done false
9561 3411 Acked Read 6 Sent Done Done false
9562 3412 Acked Read 7 Sent Done Done false
9563 3413 Acked Read 5 Sent Done Done false
9564 3414 Acked Write 9 Sent Done Done false
9565 3415 Acked Read 8 Sent Done Done false
9566 3416 Acked Read 1 Sent Done Done false
9567 3417 Acked Read 7 Sent Done Done false
9568 3418 Acked Read 7 Sent Done Done false
9569 STATES DS:0 DS:1 DS:2 TOTAL
9570 Sent 302 0 0 302
9571 Done 0 302 302 604
9572 Skipped 0 0 0 0
9573 Error 0 0 0 0
9574 Last Flush: 3116 3402 3402
9575 Downstairs last five completed: 3116 3115 3114 3113 3112
9576 Upstairs last five completed: 3418 3417 3416 3415 3414
9577 CLIENT: Up:0 ds:302 act:2
9578 [8810] got start action
9579 [8810] I am going to restart
9580 Starting downstairs at port 8810
9581 Make output file at "/tmp/dsc/downstairs-8810.txt"
9582 [8810][0] reports Starting
9583 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1207)
9584 [8810][0] reports Running
9585 ----------------------------------------------------------------
9586 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
9587 States: Offline Active Active
9588 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9589 3117 Acked Read 10 Sent Done Done false
9590 3118 Acked Write 10 Sent Done Done false
9591 3119 Acked Write 10 Sent Done Done false
9592 3120 Acked Write 2 Sent Done Done false
9593 3121 Acked Flush 0 Sent Done Done false
9594 3122 Acked Flush 0 Sent Done Done false
9595 3123 Acked Write 8 Sent Done Done false
9596 3124 Acked Read 5 Sent Done Done false
9597 3125 Acked Write 1 Sent Done Done false
9598 3126 Acked Flush 0 Sent Done Done false
9599 3127 Acked Read 10 Sent Done Done false
9600 3128 Acked Read 6 Sent Done Done false
9601 3129 Acked Read 5 Sent Done Done false
9602 3130 Acked Flush 0 Sent Done Done false
9603 3131 Acked Flush 0 Sent Done Done false
9604 3132 Acked Read 4 Sent Done Done false
9605 3133 Acked Write 1 Sent Done Done false
9606 3134 Acked Read 6 Sent Done Done false
9607 3135 Acked Flush 0 Sent Done Done false
9608 3136 Acked Write 2 Sent Done Done false
9609 3137 Acked Flush 0 Sent Done Done false
9610 3138 Acked Read 3 Sent Done Done false
9611 3139 Acked Flush 0 Sent Done Done false
9612 3140 Acked Read 7 Sent Done Done false
9613 3141 Acked Read 8 Sent Done Done false
9614 3142 Acked Write 8 Sent Done Done false
9615 3143 Acked Read 2 Sent Done Done false
9616 3144 Acked Read 2 Sent Done Done false
9617 3145 Acked Read 9 Sent Done Done false
9618 3146 Acked Write 9 Sent Done Done false
9619 3147 Acked Read 8 Sent Done Done false
9620 3148 Acked Read 5 Sent Done Done false
9621 3149 Acked Read 9 Sent Done Done false
9622 3150 Acked Read 7 Sent Done Done false
9623 3151 Acked Write 2 Sent Done Done false
9624 3152 Acked Read 5 Sent Done Done false
9625 3153 Acked Write 4 Sent Done Done false
9626 3154 Acked Read 2 Sent Done Done false
9627 3155 Acked Flush 0 Sent Done Done false
9628 3156 Acked Read 7 Sent Done Done false
9629 3157 Acked Read 6 Sent Done Done false
9630 3158 Acked Read 10 Sent Done Done false
9631 3159 Acked Flush 0 Sent Done Done false
9632 3160 Acked Read 9 Sent Done Done false
9633 3161 Acked Read 2 Sent Done Done false
9634 3162 Acked Write 3 Sent Done Done false
9635 3163 Acked Read 10 Sent Done Done false
9636 3164 Acked Flush 0 Sent Done Done false
9637 3165 Acked Flush 0 Sent Done Done false
9638 3166 Acked Flush 0 Sent Done Done false
9639 3167 Acked Write 9 Sent Done Done false
9640 3168 Acked Read 7 Sent Done Done false
9641 3169 Acked Read 3 Sent Done Done false
9642 3170 Acked Read 8 Sent Done Done false
9643 3171 Acked Read 9 Sent Done Done false
9644 3172 Acked Flush 0 Sent Done Done false
9645 3173 Acked Read 10 Sent Done Done false
9646 3174 Acked Write 2 Sent Done Done false
9647 3175 Acked Read 8 Sent Done Done false
9648 3176 Acked Write 2 Sent Done Done false
9649 3177 Acked Read 7 Sent Done Done false
9650 3178 Acked Read 4 Sent Done Done false
9651 3179 Acked Flush 0 Sent Done Done false
9652 3180 Acked Write 4 Sent Done Done false
9653 3181 Acked Read 2 Sent Done Done false
9654 3182 Acked Flush 0 Sent Done Done false
9655 3183 Acked Read 5 Sent Done Done false
9656 3184 Acked Read 2 Sent Done Done false
9657 3185 Acked Read 2 Sent Done Done false
9658 3186 Acked Read 4 Sent Done Done false
9659 3187 Acked Write 2 Sent Done Done false
9660 3188 Acked Read 7 Sent Done Done false
9661 3189 Acked Write 5 Sent Done Done false
9662 3190 Acked Flush 0 Sent Done Done false
9663 3191 Acked Write 6 Sent Done Done false
9664 3192 Acked Read 6 Sent Done Done false
9665 3193 Acked Read 9 Sent Done Done false
9666 3194 Acked Write 5 Sent Done Done false
9667 3195 Acked Write 6 Sent Done Done false
9668 3196 Acked Read 9 Sent Done Done false
9669 3197 Acked Write 8 Sent Done Done false
9670 3198 Acked Read 7 Sent Done Done false
9671 3199 Acked Read 10 Sent Done Done false
9672 3200 Acked Read 5 Sent Done Done false
9673 3201 Acked Write 9 Sent Done Done false
9674 3202 Acked Flush 0 Sent Done Done false
9675 3203 Acked Write 10 Sent Done Done false
9676 3204 Acked Read 2 Sent Done Done false
9677 3205 Acked Write 4 Sent Done Done false
9678 3206 Acked Read 4 Sent Done Done false
9679 3207 Acked Write 1 Sent Done Done false
9680 3208 Acked Read 10 Sent Done Done false
9681 3209 Acked Flush 0 Sent Done Done false
9682 3210 Acked Read 8 Sent Done Done false
9683 3211 Acked Write 7 Sent Done Done false
9684 3212 Acked Read 6 Sent Done Done false
9685 3213 Acked Write 2 Sent Done Done false
9686 3214 Acked Read 1 Sent Done Done false
9687 3215 Acked Write 10 Sent Done Done false
9688 3216 Acked Read 1 Sent Done Done false
9689 3217 Acked Write 8 Sent Done Done false
9690 3218 Acked Write 6 Sent Done Done false
9691 3219 Acked Read 7 Sent Done Done false
9692 3220 Acked Write 2 Sent Done Done false
9693 3221 Acked Read 4 Sent Done Done false
9694 3222 Acked Write 3 Sent Done Done false
9695 3223 Acked Read 9 Sent Done Done false
9696 3224 Acked Write 4 Sent Done Done false
9697 3225 Acked Write 7 Sent Done Done false
9698 3226 Acked Write 2 Sent Done Done false
9699 3227 Acked Write 10 Sent Done Done false
9700 3228 Acked Flush 0 Sent Done Done false
9701 3229 Acked Flush 0 Sent Done Done false
9702 3230 Acked Read 10 Sent Done Done false
9703 3231 Acked Flush 0 Sent Done Done false
9704 3232 Acked Read 5 Sent Done Done false
9705 3233 Acked Read 9 Sent Done Done false
9706 3234 Acked Write 5 Sent Done Done false
9707 3235 Acked Read 2 Sent Done Done false
9708 3236 Acked Flush 0 Sent Done Done false
9709 3237 Acked Write 8 Sent Done Done false
9710 3238 Acked Read 9 Sent Done Done false
9711 3239 Acked Read 4 Sent Done Done false
9712 3240 Acked Read 9 Sent Done Done false
9713 3241 Acked Read 5 Sent Done Done false
9714 3242 Acked Flush 0 Sent Done Done false
9715 3243 Acked Write 9 Sent Done Done false
9716 3244 Acked Flush 0 Sent Done Done false
9717 3245 Acked Write 9 Sent Done Done false
9718 3246 Acked Write 9 Sent Done Done false
9719 3247 Acked Write 9 Sent Done Done false
9720 3248 Acked Flush 0 Sent Done Done false
9721 3249 Acked Write 8 Sent Done Done false
9722 3250 Acked Read 3 Sent Done Done false
9723 3251 Acked Read 6 Sent Done Done false
9724 3252 Acked Read 1 Sent Done Done false
9725 3253 Acked Flush 0 Sent Done Done false
9726 3254 Acked Read 1 Sent Done Done false
9727 3255 Acked Read 4 Sent Done Done false
9728 3256 Acked Read 4 Sent Done Done false
9729 3257 Acked Flush 0 Sent Done Done false
9730 3258 Acked Read 10 Sent Done Done false
9731 3259 Acked Write 9 Sent Done Done false
9732 3260 Acked Write 2 Sent Done Done false
9733 3261 Acked Read 9 Sent Done Done false
9734 3262 Acked Read 4 Sent Done Done false
9735 3263 Acked Read 7 Sent Done Done false
9736 3264 Acked Write 3 Sent Done Done false
9737 3265 Acked Write 8 Sent Done Done false
9738 3266 Acked Flush 0 Sent Done Done false
9739 3267 Acked Write 7 Sent Done Done false
9740 3268 Acked Read 4 Sent Done Done false
9741 3269 Acked Read 2 Sent Done Done false
9742 3270 Acked Read 2 Sent Done Done false
9743 3271 Acked Write 3 Sent Done Done false
9744 3272 Acked Flush 0 Sent Done Done false
9745 3273 Acked Write 8 Sent Done Done false
9746 3274 Acked Flush 0 Sent Done Done false
9747 3275 Acked Flush 0 Sent Done Done false
9748 3276 Acked Read 10 Sent Done Done false
9749 3277 Acked Read 6 Sent Done Done false
9750 3278 Acked Write 5 Sent Done Done false
9751 3279 Acked Write 3 Sent Done Done false
9752 3280 Acked Read 3 Sent Done Done false
9753 3281 Acked Read 7 Sent Done Done false
9754 3282 Acked Read 3 Sent Done Done false
9755 3283 Acked Read 3 Sent Done Done false
9756 3284 Acked Read 7 Sent Done Done false
9757 3285 Acked Read 9 Sent Done Done false
9758 3286 Acked Write 9 Sent Done Done false
9759 3287 Acked Read 10 Sent Done Done false
9760 3288 Acked Read 9 Sent Done Done false
9761 3289 Acked Read 1 Sent Done Done false
9762 3290 Acked Read 5 Sent Done Done false
9763 3291 Acked Read 8 Sent Done Done false
9764 3292 Acked Read 6 Sent Done Done false
9765 3293 Acked Write 4 Sent Done Done false
9766 3294 Acked Read 5 Sent Done Done false
9767 3295 Acked Flush 0 Sent Done Done false
9768 3296 Acked Write 2 Sent Done Done false
9769 3297 Acked Read 7 Sent Done Done false
9770 3298 Acked Write 7 Sent Done Done false
9771 3299 Acked Write 7 Sent Done Done false
9772 3300 Acked Write 4 Sent Done Done false
9773 3301 Acked Write 10 Sent Done Done false
9774 3302 Acked Write 7 Sent Done Done false
9775 3303 Acked Read 9 Sent Done Done false
9776 3304 Acked Write 10 Sent Done Done false
9777 3305 Acked Read 8 Sent Done Done false
9778 3306 Acked Read 10 Sent Done Done false
9779 3307 Acked Write 2 Sent Done Done false
9780 3308 Acked Read 4 Sent Done Done false
9781 3309 Acked Write 10 Sent Done Done false
9782 3310 Acked Read 4 Sent Done Done false
9783 3311 Acked Read 10 Sent Done Done false
9784 3312 Acked Read 3 Sent Done Done false
9785 3313 Acked Write 7 Sent Done Done false
9786 3314 Acked Write 3 Sent Done Done false
9787 3315 Acked Write 10 Sent Done Done false
9788 3316 Acked Write 4 Sent Done Done false
9789 3317 Acked Write 1 Sent Done Done false
9790 3318 Acked Write 4 Sent Done Done false
9791 3319 Acked Read 9 Sent Done Done false
9792 3320 Acked Flush 0 Sent Done Done false
9793 3321 Acked Write 6 Sent Done Done false
9794 3322 Acked Read 8 Sent Done Done false
9795 3323 Acked Read 5 Sent Done Done false
9796 3324 Acked Write 10 Sent Done Done false
9797 3325 Acked Flush 0 Sent Done Done false
9798 3326 Acked Read 7 Sent Done Done false
9799 3327 Acked Write 5 Sent Done Done false
9800 3328 Acked Read 1 Sent Done Done false
9801 3329 Acked Write 3 Sent Done Done false
9802 3330 Acked Write 1 Sent Done Done false
9803 3331 Acked Flush 0 Sent Done Done false
9804 3332 Acked Read 7 Sent Done Done false
9805 3333 Acked Write 7 Sent Done Done false
9806 3334 Acked Flush 0 Sent Done Done false
9807 3335 Acked Read 9 Sent Done Done false
9808 3336 Acked Flush 0 Sent Done Done false
9809 3337 Acked Write 2 Sent Done Done false
9810 3338 Acked Read 2 Sent Done Done false
9811 3339 Acked Write 10 Sent Done Done false
9812 3340 Acked Read 9 Sent Done Done false
9813 3341 Acked Read 10 Sent Done Done false
9814 3342 Acked Write 4 Sent Done Done false
9815 3343 Acked Read 3 Sent Done Done false
9816 3344 Acked Read 8 Sent Done Done false
9817 3345 Acked Write 4 Sent Done Done false
9818 3346 Acked Read 8 Sent Done Done false
9819 3347 Acked Read 3 Sent Done Done false
9820 3348 Acked Read 2 Sent Done Done false
9821 3349 Acked Read 4 Sent Done Done false
9822 3350 Acked Read 5 Sent Done Done false
9823 3351 Acked Read 6 Sent Done Done false
9824 3352 Acked Read 6 Sent Done Done false
9825 3353 Acked Write 2 Sent Done Done false
9826 3354 Acked Write 9 Sent Done Done false
9827 3355 Acked Write 2 Sent Done Done false
9828 3356 Acked Read 10 Sent Done Done false
9829 3357 Acked Write 2 Sent Done Done false
9830 3358 Acked Read 8 Sent Done Done false
9831 3359 Acked Write 8 Sent Done Done false
9832 3360 Acked Write 1 Sent Done Done false
9833 3361 Acked Read 6 Sent Done Done false
9834 3362 Acked Write 2 Sent Done Done false
9835 3363 Acked Read 1 Sent Done Done false
9836 3364 Acked Flush 0 Sent Done Done false
9837 3365 Acked Write 3 Sent Done Done false
9838 3366 Acked Read 6 Sent Done Done false
9839 3367 Acked Write 1 Sent Done Done false
9840 3368 Acked Write 6 Sent Done Done false
9841 3369 Acked Write 6 Sent Done Done false
9842 3370 Acked Read 1 Sent Done Done false
9843 3371 Acked Read 9 Sent Done Done false
9844 3372 Acked Read 8 Sent Done Done false
9845 3373 Acked Read 9 Sent Done Done false
9846 3374 Acked Flush 0 Sent Done Done false
9847 3375 Acked Write 6 Sent Done Done false
9848 3376 Acked Write 5 Sent Done Done false
9849 3377 Acked Flush 0 Sent Done Done false
9850 3378 Acked Write 4 Sent Done Done false
9851 3379 Acked Read 2 Sent Done Done false
9852 3380 Acked Write 8 Sent Done Done false
9853 3381 Acked Write 4 Sent Done Done false
9854 3382 Acked Read 2 Sent Done Done false
9855 3383 Acked Flush 0 Sent Done Done false
9856 3384 Acked Flush 0 Sent Done Done false
9857 3385 Acked Read 7 Sent Done Done false
9858 3386 Acked Write 3 Sent Done Done false
9859 3387 Acked Write 4 Sent Done Done false
9860 3388 Acked Flush 0 Sent Done Done false
9861 3389 Acked Read 8 Sent Done Done false
9862 3390 Acked Write 4 Sent Done Done false
9863 3391 Acked Write 6 Sent Done Done false
9864 3392 Acked Read 8 Sent Done Done false
9865 3393 Acked Read 7 Sent Done Done false
9866 3394 Acked Write 1 Sent Done Done false
9867 3395 Acked Write 4 Sent Done Done false
9868 3396 Acked Write 4 Sent Done Done false
9869 3397 Acked Write 2 Sent Done Done false
9870 3398 Acked Read 8 Sent Done Done false
9871 3399 Acked Write 5 Sent Done Done false
9872 3400 Acked Read 5 Sent Done Done false
9873 3401 Acked Read 10 Sent Done Done false
9874 3402 Acked Flush 0 Sent Done Done false
9875 3403 Acked Write 1 Sent Done Done false
9876 3404 Acked Read 9 Sent Done Done false
9877 3405 Acked Read 2 Sent Done Done false
9878 3406 Acked Read 1 Sent Done Done false
9879 3407 Acked Read 3 Sent Done Done false
9880 3408 Acked Read 8 Sent Done Done false
9881 3409 Acked Write 7 Sent Done Done false
9882 3410 Acked Write 6 Sent Done Done false
9883 3411 Acked Read 6 Sent Done Done false
9884 3412 Acked Read 7 Sent Done Done false
9885 3413 Acked Read 5 Sent Done Done false
9886 3414 Acked Write 9 Sent Done Done false
9887 3415 Acked Read 8 Sent Done Done false
9888 3416 Acked Read 1 Sent Done Done false
9889 3417 Acked Read 7 Sent Done Done false
9890 3418 Acked Read 7 Sent Done Done false
9891 3419 Acked Flush 0 Sent Done Done false
9892 STATES DS:0 DS:1 DS:2 TOTAL
9893 Sent 303 0 0 303
9894 Done 0 303 303 606
9895 Skipped 0 0 0 0
9896 Error 0 0 0 0
9897 Last Flush: 3116 3419 3419
9898 Downstairs last five completed: 3116 3115 3114 3113 3112
9899 Upstairs last five completed: 3419 3418 3417 3416 3415
9900 CLIENT: Up:0 ds:303 act:2
99012024-11-07T14:05:21.877ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99022024-11-07T14:05:21.877ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41172) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99032024-11-07T14:05:21.878ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99042024-11-07T14:05:21.879ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99052024-11-07T14:05:21.879ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99062024-11-07T14:05:21.879ZINFOcrucible: send last flush ID to this DS: 3116 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99072024-11-07T14:05:21.879ZINFOcrucible: Replied this last flush ID: 3116 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99082024-11-07T14:05:21.879ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99092024-11-07T14:05:21.879ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99102024-11-07T14:05:21.879ZINFOcrucible: [0] client re-new 303 jobs since flush 3116 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99112024-11-07T14:05:21.879ZINFOcrucible: [0] Replayed 303 jobs since flush: 3116 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
9912 ----------------------------------------------------------------
9913 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9914 Downstairs last five completed: 3419 3418 3417 3416 3415
9915 Upstairs last five completed: 3419 3418 3417 3416 3415
9916 CLIENT: Up:0 ds:0 act:3
9917 Replay: All jobs finished
9918 Main task has work to do, go find it
9919 got dsc Stop(2)
9920 stop 2
9921 Nov 07 14:05:24.184 INFO request completed, latency_us[8830] Got stop action so:false kr:false
9922 : 343, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 300a84c8-67ff-44f2-a16f-435a5df50f9c, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
99232024-11-07T14:05:24.185ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
9924 Nov 07 14:05:24.185 INFO request completed, latency_us: 169, response_code: 200, uri: /state/cid/2, method: GET, req_id: 32ed61d6-e829-45a7-891a-a7d8f5ba9fb3{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-07T14:05:24.185457738Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"":"io task","client":"2","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
9925 , remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
99262024-11-07T14:05:24.185ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99272024-11-07T14:05:24.185ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99282024-11-07T14:05:24.185ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
99292024-11-07T14:05:24.185ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
9930 [8830] Exited with: ExitStatus(unix_wait_status(9))
9931 [8830][2] reports Exit
9932 Nov 07 14:05:28.187 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/2, method: GET, req_id: 568d6730-9f1d-49b0-8295-391883ae76c0, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
9933 001/300 Read block 2967 len 40960
9934 002/300 Write block 2475 len 36864 data: 5 5 4 3 3 2 2 2 2
9935 003/300 Read block 617 len 32768
9936 004/300 Write block 2106 len 8192 data: 2 2
9937 005/300 Read block 2563 len 8192
9938 006/300 Write block 403 len 16384 data: 4 4 4 3
9939 007/300 Write block 4757 len 36864 data: 4 4 3 3 2 2 2 2 3
9940 008/300 Write block 2454 len 24576 data: 2 2 2 2 2 2
9941 009/300 Write block 1171 len 40960 data: 4 4 4 4 3 3 3 3 2 2
9942 010/300 Write block 218 len 12288 data: 2 2 2
9943 011/300 Read block 338 len 28672
9944 012/300 Read block 4802 len 20480
9945 013/300 Write block 3076 len 12288 data: 3 2 2
9946 014/300 Write block 2233 len 36864 data: 4 3 2 3 3 3 3 3 3
9947 015/300 Read block 1412 len 36864
9948 016/300 Write block 4984 len 32768 data: 3 3 3 3 3 4 3 3
9949 017/300 Write block 4439 len 28672 data: 4 3 3 2 2 2 2
9950 018/300 Read block 2867 len 20480
9951 019/300 Read block 2938 len 8192
9952 020/300 Read block 128 len 4096
9953 021/300 Read block 4245 len 24576
9954 022/300 Read block 3047 len 8192
9955 023/300 Read block 3510 len 32768
9956 024/300 Read block 369 len 24576
9957 025/300 Flush
9958 026/300 Write block 3462 len 12288 data: 3 2 2
9959 027/300 Read block 2034 len 8192
9960 028/300 Write block 3862 len 16384 data: 4 4 4 4
9961 029/300 Read block 2493 len 20480
9962 030/300 Flush
9963 031/300 Read block 1253 len 8192
9964 032/300 Write block 1767 len 16384 data: 2 3 4 4
9965 033/300 Write block 4537 len 24576 data: 3 3 3 2 3 3
9966 034/300 Write block 4709 len 20480 data: 2 2 2 3 3
9967 035/300 Write block 3114 len 12288 data: 2 3 3
9968 036/300 Write block 2549 len 8192 data: 4 3
9969 037/300 Write block 2969 len 20480 data: 3 3 3 3 3
9970 038/300 Write block 4401 len 16384 data: 2 2 2 2
9971 039/300 Read block 727 len 36864
9972 040/300 Read block 1384 len 20480
9973 041/300 Read block 576 len 16384
9974 042/300 Read block 4592 len 36864
9975 043/300 Write block 2139 len 4096 data: 3
9976 044/300 Read block 1164 len 28672
9977 045/300 Write block 4746 len 4096 data: 3
9978 046/300 Write block 2761 len 8192 data: 2 3
9979 047/300 Write block 4185 len 4096 data: 2
9980 048/300 Read block 1597 len 24576
9981 049/300 Read block 2677 len 8192
9982 050/300 Write block 3198 len 24576 data: 2 2 2 2 2 2
9983 051/300 Read block 286 len 32768
9984 052/300 Write block 148 len 4096 data: 3
9985 053/300 Write block 2730 len 36864 data: 2 2 2 2 2 2 2 3 3
9986 054/300 Read block 2867 len 20480
9987 055/300 Write block 3323 len 28672 data: 4 4 4 4 3 4 4
9988 056/300 Write block 707 len 12288 data: 6 5 4
9989 057/300 Read block 1222 len 28672
9990 058/300 Read block 1556 len 24576
9991 059/300 Write block 272 len 32768 data: 2 2 2 3 2 2 2 2
9992 060/300 Flush
9993 061/300 Write block 2380 len 20480 data: 2 2 2 2 3
9994 062/300 Read block 4045 len 16384
9995 063/300 Read block 1844 len 16384
9996 064/300 Flush
9997 065/300 Read block 4186 len 20480
9998 066/300 Read block 3206 len 40960
9999 067/300 Read block 3474 len 32768
10000 068/300 Read block 3592 len 32768
10001 069/300 Read block 553 len 4096
10002 070/300 Read block 2267 len 32768
10003 071/300 Write block 3921 len 16384 data: 2 2 3 4
10004 072/300 Read block 1611 len 4096
10005 073/300 Write block 2910 len 28672 data: 3 4 4 3 3 2 3
10006 074/300 Read block 1829 len 8192
10007 075/300 Read block 1892 len 28672
10008 076/300 Read block 3663 len 16384
10009 077/300 Read block 221 len 20480
10010 078/300 Read block 4065 len 12288
10011 079/300 Write block 4047 len 36864 data: 3 3 2 2 2 2 2 2 2
10012 080/300 Write block 3049 len 20480 data: 3 3 3 3 4
10013 081/300 Read block 2772 len 4096
10014 082/300 Write block 3797 len 32768 data: 5 5 5 5 4 4 4 4
10015 083/300 Read block 317 len 12288
10016 084/300 Read block 1963 len 4096
10017 085/300 Write block 4744 len 32768 data: 3 3 4 2 2 2 3 3
10018 086/300 Flush
10019 087/300 Read block 797 len 24576
10020 088/300 Flush
10021 089/300 Read block 4098 len 40960
10022 090/300 Read block 2807 len 36864
10023 091/300 Write block 771 len 24576 data: 2 2 2 2 2 2
10024 092/300 Read block 323 len 4096
10025 093/300 Write block 3522 len 4096 data: 2
10026 094/300 Read block 3027 len 20480
10027 095/300 Read block 2842 len 12288
10028 096/300 Read block 3233 len 8192
10029 097/300 Read block 4872 len 4096
10030 098/300 Read block 576 len 24576
10031 099/300 Read block 2544 len 16384
10032 100/300 Read block 4930 len 40960
10033 101/300 Read block 3981 len 32768
10034 102/300 Write block 4569 len 8192 data: 2 3
10035 103/300 Read block 3353 len 20480
10036 104/300 Read block 4683 len 4096
10037 105/300 Write block 1647 len 24576 data: 5 5 4 4 4 4
10038 106/300 Write block 864 len 40960 data: 3 3 3 3 3 3 4 3 4 4
10039 107/300 Read block 3096 len 28672
10040 108/300 Flush
10041 109/300 Read block 4056 len 12288
10042 110/300 Write block 3411 len 32768 data: 2 2 2 2 2 2 2 2
10043 111/300 Write block 3998 len 8192 data: 3 2
10044 112/300 Write block 1653 len 20480 data: 4 4 4 3 3
10045 113/300 Write block 3530 len 40960 data: 2 3 3 4 4 4 3 3 3 3
10046 114/300 Write block 1877 len 40960 data: 3 3 3 2 2 2 3 4 3 3
10047 115/300 Write block 1365 len 40960 data: 3 3 2 2 2 2 2 2 2 4
10048 116/300 Read block 2688 len 24576
10049 117/300 Read block 155 len 32768
10050 118/300 Write block 4892 len 20480 data: 2 2 2 3 3
10051 119/300 Read block 1204 len 28672
10052 120/300 Read block 2696 len 40960
10053 121/300 Write block 2694 len 28672 data: 3 3 3 3 3 3 3
10054 122/300 Flush
10055 123/300 Read block 2108 len 28672
10056 124/300 Read block 3884 len 12288
10057 125/300 Write block 465 len 24576 data: 2 3 3 4 4 4
10058 126/300 Read block 913 len 8192
10059 127/300 Write block 531 len 40960 data: 3 3 3 2 2 2 2 2 2 2
10060 128/300 Read block 4360 len 4096
10061 129/300 Read block 1416 len 20480
10062 130/300 Write block 2239 len 16384 data: 4 4 4 2
10063 131/300 Write block 3757 len 32768 data: 2 2 2 2 3 3 3 3
10064 132/300 Read block 3924 len 12288
10065 133/300 Read block 1252 len 32768
10066 134/300 Read block 1568 len 28672
10067 135/300 Write block 473 len 20480 data: 2 2 2 2 3
10068 136/300 Read block 227 len 16384
10069 137/300 Write block 4915 len 12288 data: 3 3 3
10070 138/300 Write block 928 len 4096 data: 2
10071 139/300 Read block 4208 len 24576
10072 140/300 Flush
10073 141/300 Write block 4779 len 16384 data: 3 3 3 2
10074 142/300 Read block 4954 len 8192
10075 143/300 Read block 1075 len 28672
10076 144/300 Write block 838 len 4096 data: 2
10077 145/300 Read block 4474 len 24576
10078 146/300 Read block 2393 len 28672
10079 147/300 Read block 717 len 4096
10080 148/300 Flush
10081 149/300 Write block 913 len 36864 data: 3 4 4 4 4 4 5 3 2
10082 150/300 Write block 91 len 36864 data: 3 3 3 2 2 2 2 2 3
10083 151/300 Write block 1749 len 20480 data: 3 4 4 3 3
10084 152/300 Read block 3986 len 32768
10085 153/300 Write block 4452 len 32768 data: 3 3 2 2 2 2 2 2
10086 154/300 Write block 3066 len 20480 data: 3 3 3 3 2
10087 155/300 Write block 2404 len 8192 data: 2 3
10088 156/300 Read block 4002 len 32768
10089 157/300 Read block 4445 len 12288
10090 158/300 Write block 1828 len 20480 data: 2 2 2 2 2
10091 159/300 Write block 452 len 4096 data: 4
10092 160/300 Read block 1098 len 36864
10093 161/300 Read block 880 len 20480
10094 162/300 Flush
10095 163/300 Read block 3108 len 36864
10096 164/300 Read block 1184 len 24576
10097 165/300 Read block 4911 len 32768
10098 166/300 Read block 1380 len 28672
10099 167/300 Read block 4315 len 12288
10100 168/300 Read block 3148 len 12288
10101 169/300 Read block 3701 len 16384
10102 170/300 Write block 3092 len 8192 data: 4 3
10103 171/300 Write block 2964 len 8192 data: 2 2
10104 172/300 Read block 2228 len 12288
10105 173/300 Write block 2592 len 24576 data: 3 3 3 3 3 3
10106 174/300 Read block 4722 len 40960
10107 175/300 Write block 2683 len 16384 data: 2 2 2 2
10108 176/300 Write block 4314 len 4096 data: 5
10109 177/300 Read block 2010 len 20480
10110 178/300 Flush
10111 179/300 Write block 4296 len 32768 data: 3 3 4 4 4 3 3 3
10112 180/300 Write block 4171 len 32768 data: 3 3 4 4 3 3 3 3
10113 181/300 Read block 2962 len 12288
10114 182/300 Write block 1043 len 36864 data: 4 4 4 4 4 4 4 4 4
10115 183/300 Write block 3587 len 20480 data: 2 2 2 2 2
10116 184/300 Flush
10117 185/300 Write block 2480 len 24576 data: 3 3 3 3 2 3
10118 186/300 Read block 682 len 12288
10119 187/300 Flush
10120 188/300 Write block 3278 len 32768 data: 3 3 3 2 4 4 3 4
10121 189/300 Read block 792 len 12288
10122 190/300 Write block 2442 len 20480 data: 3 3 3 3 4
10123 191/300 Read block 1151 len 4096
10124 192/300 Write block 752 len 24576 data: 5 4 4 3 3 2
10125 193/300 Read block 1823 len 12288
10126 194/300 Flush
10127 195/300 Read block 4528 len 12288
10128 196/300 Read block 3009 len 32768
10129 197/300 Flush
10130 198/300 Flush
10131 199/300 Flush
10132 200/300 Read block 2224 len 4096
10133 201/300 Write block 2880 len 24576 data: 2 2 3 3 3 3
10134 202/300 Read block 28 len 4096
10135 203/300 Read block 3984 len 24576
10136 204/300 Read block 4832 len 32768
10137 205/300 Write block 4759 len 32768 data: 4 4 3 3 3 3 4 3
10138 206/300 Read block 1037 len 40960
10139 207/300 Read block 377 len 20480
10140 208/300 Read block 2869 len 28672
10141 209/300 Read block 643 len 20480
10142 210/300 Read block 587 len 32768
10143 211/300 Write block 3030 len 40960 data: 2 2 2 2 2 2 2 2 3 3
10144 212/300 Read block 4016 len 24576
10145 213/300 Read block 1137 len 8192
10146 214/300 Read block 315 len 36864
10147 215/300 Write block 2150 len 16384 data: 2 2 2 2
10148 216/300 Read block 3642 len 4096
10149 217/300 Write block 1317 len 24576 data: 2 2 2 2 2 2
10150 218/300 Read block 2570 len 20480
10151 219/300 Write block 3861 len 16384 data: 4 5 5 5
10152 220/300 Write block 4382 len 40960 data: 2 3 4 4 3 3 3 3 3 3
10153 221/300 Read block 1262 len 36864
10154 222/300 Flush
10155 223/300 Flush
10156 224/300 Flush
10157 225/300 Read block 4583 len 16384
10158 226/300 Write block 3080 len 28672 data: 2 4 4 4 4 3 3
10159 227/300 Flush
10160 228/300 Write block 1738 len 8192 data: 2 2
10161 229/300 Read block 949 len 40960
10162 230/300 Read block 2824 len 8192
10163 231/300 Read block 3153 len 16384
10164 232/300 Read block 1799 len 32768
10165 233/300 Flush
10166 234/300 Write block 4919 len 16384 data: 4 4 3 3
10167 235/300 Write block 450 len 36864 data: 3 3 5 4 5 4 5 5 3
10168 236/300 Write block 762 len 8192 data: 3 3
10169 237/300 Read block 4881 len 40960
10170 238/300 Write block 4928 len 24576 data: 3 3 3 3 3 4
10171 239/300 Write block 4210 len 28672 data: 2 2 2 2 2 2 2
10172 240/300 Write block 1853 len 8192 data: 6 6
10173 241/300 Read block 2006 len 8192
10174 242/300 Write block 3551 len 24576 data: 3 4 4 3 3 2
10175 243/300 Read block 4732 len 24576
10176 244/300 Read block 313 len 20480
10177 245/300 Flush
10178 246/300 Write block 4860 len 20480 data: 2 2 2 2 3
10179 247/300 Read block 3927 len 32768
10180 248/300 Flush
10181 249/300 Write block 4949 len 40960 data: 2 3 3 4 4 4 4 4 4 3
10182 250/300 Write block 442 len 4096 data: 4
10183 251/300 Read block 4122 len 16384
10184 252/300 Read block 3790 len 20480
10185 253/300 Write block 4896 len 20480 data: 4 3 3 3 3
10186 254/300 Write block 758 len 20480 data: 2 2 2 2 4
10187 255/300 Flush
10188 256/300 Read block 4971 len 32768
10189 257/300 Read block 752 len 36864
10190 258/300 Read block 2783 len 20480
10191 259/300 Read block 3629 len 4096
10192 260/300 Read block 5 len 8192
10193 261/300 Write block 2494 len 12288 data: 4 4 4
10194 262/300 Read block 118 len 32768
10195 263/300 Read block 2968 len 16384
10196 264/300 Read block 1542 len 8192
10197 265/300 Write block 2660 len 20480 data: 3 3 3 4 4
10198 266/300 Flush
10199 267/300 Flush
10200 268/300 Read block 1394 len 4096
10201 269/300 Read block 2581 len 28672
10202 270/300 Read block 3521 len 40960
10203 271/300 Read block 3770 len 28672
10204 272/300 Flush
10205 273/300 Read block 4728 len 36864
10206 274/300 Write block 3889 len 36864 data: 3 3 3 3 2 2 4 4 4
10207 275/300 Read block 2221 len 12288
10208 276/300 Write block 4161 len 16384 data: 3 3 2 2
10209 277/300 Read block 4617 len 4096
10210 278/300 Write block 2158 len 12288 data: 4 3 2
10211 279/300 Read block 3811 len 28672
10212 280/300 Write block 2388 len 4096 data: 3
10213 281/300 Read block 1488 len 12288
10214 282/300 Flush
10215 283/300 Read block 3811 len 32768
10216 284/300 Write block 2895 len 8192 data: 2 2
10217 285/300 Read block 3615 len 40960
10218 286/300 Flush
10219 287/300 Flush
10220 288/300 Read block 882 len 12288
10221 289/300 Read block 550 len 28672
10222 290/300 Read block 3620 len 40960
10223 291/300 Read block 2090 len 20480
10224 292/300 Read block 3456 len 8192
10225 293/300 Read block 2036 len 28672
10226 294/300 Flush
10227 295/300 Read block 3314 len 8192
10228 296/300 Read block 4017 len 36864
10229 297/300 Write block 1873 len 28672 data: 2 2 3 3 4 4 4
10230 298/300 Flush
10231 299/300 Read block 4693 len 20480
10232 300/300 Write block 848 len 36864 data: 3 3 3 3 3 2 3 3 2
10233 301/300 Write block 1852 len 40960 data: 6 7 7 5 5 4 4 3 3 3
10234 Main task has work to do, go find it
10235 got dsc Start(2)
10236 start 2
10237 Nov 07 14:05:28.462 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/2, method: POST, req_id: 94a46a79-381e-4354-bb1f-c584384b683c, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
10238 Replay: started 2, returned:Ok(())
10239 ----------------------------------------------------------------
10240 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10241 States: Active Active Offline
10242 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10243 3420 Acked Read 10 Done Done Sent false
10244 3421 Acked Write 9 Done Done Sent false
10245 3422 Acked Read 8 Done Done Sent false
10246 3423 Acked Write 2 Done Done Sent false
10247 3424 Acked Read 2 Done Done Sent false
10248 3425 Acked Write 4 Done Done Sent false
10249 3426 Acked Write 9 Done Done Sent false
10250 3427 Acked Write 6 Done Done Sent false
10251 3428 Acked Write 10 Done Done Sent false
10252 3429 Acked Write 3 Done Done Sent false
10253 3430 Acked Read 7 Done Done Sent false
10254 3431 Acked Read 5 Done Done Sent false
10255 3432 Acked Write 3 Done Done Sent false
10256 3433 Acked Write 9 Done Done Sent false
10257 3434 Acked Read 9 Done Done Sent false
10258 3435 Acked Write 8 Done Done Sent false
10259 3436 Acked Write 7 Done Done Sent false
10260 3437 Acked Read 5 Done Done Sent false
10261 3438 Acked Read 2 Done Done Sent false
10262 3439 Acked Read 1 Done Done Sent false
10263 3440 Acked Read 6 Done Done Sent false
10264 3441 Acked Read 2 Done Done Sent false
10265 3442 Acked Read 8 Done Done Sent false
10266 3443 Acked Read 6 Done Done Sent false
10267 3444 Acked Flush 0 Done Done Sent false
10268 3445 Acked Write 3 Done Done Sent false
10269 3446 Acked Read 2 Done Done Sent false
10270 3447 Acked Write 4 Done Done Sent false
10271 3448 Acked Read 5 Done Done Sent false
10272 3449 Acked Flush 0 Done Done Sent false
10273 3450 Acked Read 2 Done Done Sent false
10274 3451 Acked Write 4 Done Done Sent false
10275 3452 Acked Write 6 Done Done Sent false
10276 3453 Acked Write 5 Done Done Sent false
10277 3454 Acked Write 3 Done Done Sent false
10278 3455 Acked Write 2 Done Done Sent false
10279 3456 Acked Write 5 Done Done Sent false
10280 3457 Acked Write 4 Done Done Sent false
10281 3458 Acked Read 9 Done Done Sent false
10282 3459 Acked Read 5 Done Done Sent false
10283 3460 Acked Read 4 Done Done Sent false
10284 3461 Acked Read 9 Done Done Sent false
10285 3462 Acked Write 1 Done Done Sent false
10286 3463 Acked Read 7 Done Done Sent false
10287 3464 Acked Write 1 Done Done Sent false
10288 3465 Acked Write 2 Done Done Sent false
10289 3466 Acked Write 1 Done Done Sent false
10290 3467 Acked Read 6 Done Done Sent false
10291 3468 Acked Read 2 Done Done Sent false
10292 3469 Acked Write 6 Done Done Sent false
10293 3470 Acked Read 8 Done Done Sent false
10294 3471 Acked Write 1 Done Done Sent false
10295 3472 Acked Write 9 Done Done Sent false
10296 3473 Acked Read 5 Done Done Sent false
10297 3474 Acked Write 7 Done Done Sent false
10298 3475 Acked Write 3 Done Done Sent false
10299 3476 Acked Read 7 Done Done Sent false
10300 3477 Acked Read 6 Done Done Sent false
10301 3478 Acked Write 8 Done Done Sent false
10302 3479 Acked Flush 0 Done Done Sent false
10303 3480 Acked Write 5 Done Done Sent false
10304 3481 Acked Read 4 Done Done Sent false
10305 3482 Acked Read 4 Done Done Sent false
10306 3483 Acked Flush 0 Done Done Sent false
10307 3484 Acked Read 5 Done Done Sent false
10308 3485 Acked Read 10 Done Done Sent false
10309 3486 Acked Read 8 Done Done Sent false
10310 3487 Acked Read 8 Done Done Sent false
10311 3488 Acked Read 1 Done Done Sent false
10312 3489 Acked Read 8 Done Done Sent false
10313 3490 Acked Write 4 Done Done Sent false
10314 3491 Acked Read 1 Done Done Sent false
10315 3492 Acked Write 7 Done Done Sent false
10316 3493 Acked Read 2 Done Done Sent false
10317 3494 Acked Read 7 Done Done Sent false
10318 3495 Acked Read 4 Done Done Sent false
10319 3496 Acked Read 5 Done Done Sent false
10320 3497 Acked Read 3 Done Done Sent false
10321 3498 Acked Write 9 Done Done Sent false
10322 3499 Acked Write 5 Done Done Sent false
10323 3500 Acked Read 1 Done Done Sent false
10324 3501 Acked Write 8 Done Done Sent false
10325 3502 Acked Read 3 Done Done Sent false
10326 3503 Acked Read 1 Done Done Sent false
10327 3504 Acked Write 8 Done Done Sent false
10328 3505 Acked Flush 0 Done Done Sent false
10329 3506 Acked Read 6 Done Done Sent false
10330 3507 Acked Flush 0 Done Done Sent false
10331 3508 Acked Read 10 Done Done Sent false
10332 3509 Acked Read 9 Done Done Sent false
10333 3510 Acked Write 6 Done Done Sent false
10334 3511 Acked Read 1 Done Done Sent false
10335 3512 Acked Write 1 Done Done Sent false
10336 3513 Acked Read 5 Done Done Sent false
10337 3514 Acked Read 3 Done Done Sent false
10338 3515 Acked Read 2 Done Done Sent false
10339 3516 Acked Read 1 Done Done Sent false
10340 3517 Acked Read 6 Done Done Sent false
10341 3518 Acked Read 4 Done Done Sent false
10342 3519 Acked Read 10 Done Done Sent false
10343 3520 Acked Read 8 Done Done Sent false
10344 3521 Acked Write 2 Done Done Sent false
10345 3522 Acked Read 5 Done Done Sent false
10346 3523 Acked Read 1 Done Done Sent false
10347 3524 Acked Write 6 Done Done Sent false
10348 3525 Acked Write 10 Done Done Sent false
10349 3526 Acked Read 7 Done Done Sent false
10350 3527 Acked Flush 0 Done Done Sent false
10351 3528 Acked Read 3 Done Done Sent false
10352 3529 Acked Write 8 Done Done Sent false
10353 3530 Acked Write 2 Done Done Sent false
10354 3531 Acked Write 5 Done Done Sent false
10355 3532 Acked Write 10 Done Done Sent false
10356 3533 Acked Write 10 Done Done Sent false
10357 3534 Acked Write 10 Done Done Sent false
10358 3535 Acked Read 6 Done Done Sent false
10359 3536 Acked Read 8 Done Done Sent false
10360 3537 Acked Write 5 Done Done Sent false
10361 3538 Acked Read 7 Done Done Sent false
10362 3539 Acked Read 10 Done Done Sent false
10363 3540 Acked Write 7 Done Done Sent false
10364 3541 Acked Flush 0 Done Done Sent false
10365 3542 Acked Read 7 Done Done Sent false
10366 3543 Acked Read 3 Done Done Sent false
10367 3544 Acked Write 6 Done Done Sent false
10368 3545 Acked Read 2 Done Done Sent false
10369 3546 Acked Write 10 Done Done Sent false
10370 3547 Acked Read 1 Done Done Sent false
10371 3548 Acked Read 5 Done Done Sent false
10372 3549 Acked Write 4 Done Done Sent false
10373 3550 Acked Write 8 Done Done Sent false
10374 3551 Acked Read 3 Done Done Sent false
10375 3552 Acked Read 8 Done Done Sent false
10376 3553 Acked Read 7 Done Done Sent false
10377 3554 Acked Write 5 Done Done Sent false
10378 3555 Acked Read 4 Done Done Sent false
10379 3556 Acked Write 3 Done Done Sent false
10380 3557 Acked Write 1 Done Done Sent false
10381 3558 Acked Read 6 Done Done Sent false
10382 3559 Acked Flush 0 Done Done Sent false
10383 3560 Acked Write 4 Done Done Sent false
10384 3561 Acked Read 2 Done Done Sent false
10385 3562 Acked Read 7 Done Done Sent false
10386 3563 Acked Write 1 Done Done Sent false
10387 3564 Acked Read 6 Done Done Sent false
10388 3565 Acked Read 7 Done Done Sent false
10389 3566 Acked Read 1 Done Done Sent false
10390 3567 Acked Flush 0 Done Done Sent false
10391 3568 Acked Write 9 Done Done Sent false
10392 3569 Acked Write 9 Done Done Sent false
10393 3570 Acked Write 5 Done Done Sent false
10394 3571 Acked Read 8 Done Done Sent false
10395 3572 Acked Write 8 Done Done Sent false
10396 3573 Acked Write 5 Done Done Sent false
10397 3574 Acked Write 2 Done Done Sent false
10398 3575 Acked Read 8 Done Done Sent false
10399 3576 Acked Read 3 Done Done Sent false
10400 3577 Acked Write 5 Done Done Sent false
10401 3578 Acked Write 1 Done Done Sent false
10402 3579 Acked Read 9 Done Done Sent false
10403 3580 Acked Read 5 Done Done Sent false
10404 3581 Acked Flush 0 Done Done Sent false
10405 3582 Acked Read 9 Done Done Sent false
10406 3583 Acked Read 6 Done Done Sent false
10407 3584 Acked Read 8 Done Done Sent false
10408 3585 Acked Read 7 Done Done Sent false
10409 3586 Acked Read 3 Done Done Sent false
10410 3587 Acked Read 3 Done Done Sent false
10411 3588 Acked Read 4 Done Done Sent false
10412 3589 Acked Write 2 Done Done Sent false
10413 3590 Acked Write 2 Done Done Sent false
10414 3591 Acked Read 3 Done Done Sent false
10415 3592 Acked Write 6 Done Done Sent false
10416 3593 Acked Read 10 Done Done Sent false
10417 3594 Acked Write 4 Done Done Sent false
10418 3595 Acked Write 1 Done Done Sent false
10419 3596 Acked Read 5 Done Done Sent false
10420 3597 Acked Flush 0 Done Done Sent false
10421 3598 Acked Write 8 Done Done Sent false
10422 3599 Acked Write 8 Done Done Sent false
10423 3600 Acked Read 3 Done Done Sent false
10424 3601 Acked Write 9 Done Done Sent false
10425 3602 Acked Write 5 Done Done Sent false
10426 3603 Acked Flush 0 Done Done Sent false
10427 3604 Acked Write 6 Done Done Sent false
10428 3605 Acked Read 3 Done Done Sent false
10429 3606 Acked Flush 0 Done Done Sent false
10430 3607 Acked Write 8 Done Done Sent false
10431 3608 Acked Read 3 Done Done Sent false
10432 3609 Acked Write 5 Done Done Sent false
10433 3610 Acked Read 1 Done Done Sent false
10434 3611 Acked Write 6 Done Done Sent false
10435 3612 Acked Read 3 Done Done Sent false
10436 3613 Acked Flush 0 Done Done Sent false
10437 3614 Acked Read 3 Done Done Sent false
10438 3615 Acked Read 8 Done Done Sent false
10439 3616 Acked Flush 0 Done Done Sent false
10440 3617 Acked Flush 0 Done Done Sent false
10441 3618 Acked Flush 0 Done Done Sent false
10442 3619 Acked Read 1 Done Done Sent false
10443 3620 Acked Write 6 Done Done Sent false
10444 3621 Acked Read 1 Done Done Sent false
10445 3622 Acked Read 6 Done Done Sent false
10446 3623 Acked Read 8 Done Done Sent false
10447 3624 Acked Write 8 Done Done Sent false
10448 3625 Acked Read 10 Done Done Sent false
10449 3626 Acked Read 5 Done Done Sent false
10450 3627 Acked Read 7 Done Done Sent false
10451 3628 Acked Read 5 Done Done Sent false
10452 3629 Acked Read 8 Done Done Sent false
10453 3630 Acked Write 10 Done Done Sent false
10454 3631 Acked Read 6 Done Done Sent false
10455 3632 Acked Read 2 Done Done Sent false
10456 3633 Acked Read 9 Done Done Sent false
10457 3634 Acked Write 4 Done Done Sent false
10458 3635 Acked Read 1 Done Done Sent false
10459 3636 Acked Write 6 Done Done Sent false
10460 3637 Acked Read 5 Done Done Sent false
10461 3638 Acked Write 4 Done Done Sent false
10462 3639 Acked Write 10 Done Done Sent false
10463 3640 Acked Read 9 Done Done Sent false
10464 3641 Acked Flush 0 Done Done Sent false
10465 3642 Acked Flush 0 Done Done Sent false
10466 3643 Acked Flush 0 Done Done Sent false
10467 3644 Acked Read 4 Done Done Sent false
10468 3645 Acked Write 7 Done Done Sent false
10469 3646 Acked Flush 0 Done Done Sent false
10470 3647 Acked Write 2 Done Done Sent false
10471 3648 Acked Read 10 Done Done Sent false
10472 3649 Acked Read 2 Done Done Sent false
10473 3650 Acked Read 4 Done Done Sent false
10474 3651 Acked Read 8 Done Done Sent false
10475 3652 Acked Flush 0 Done Done Sent false
10476 3653 Acked Write 4 Done Done Sent false
10477 3654 Acked Write 9 Done Done Sent false
10478 3655 Acked Write 2 Done Done Sent false
10479 3656 Acked Read 10 Done Done Sent false
10480 3657 Acked Write 6 Done Done Sent false
10481 3658 Acked Write 7 Done Done Sent false
10482 3659 Acked Write 2 Done Done Sent false
10483 3660 Acked Read 2 Done Done Sent false
10484 3661 Acked Write 6 Done Done Sent false
10485 3662 Acked Read 6 Done Done Sent false
10486 3663 Acked Read 5 Done Done Sent false
10487 3664 Acked Flush 0 Done Done Sent false
10488 3665 Acked Write 5 Done Done Sent false
10489 3666 Acked Read 8 Done Done Sent false
10490 3667 Acked Flush 0 Done Done Sent false
10491 3668 Acked Write 10 Done Done Sent false
10492 3669 Acked Write 1 Done Done Sent false
10493 3670 Acked Read 4 Done Done Sent false
10494 3671 Acked Read 5 Done Done Sent false
10495 3672 Acked Write 5 Done Done Sent false
10496 3673 Acked Write 5 Done Done Sent false
10497 3674 Acked Flush 0 Done Done Sent false
10498 3675 Acked Read 8 Done Done Sent false
10499 3676 Acked Read 9 Done Done Sent false
10500 3677 Acked Read 5 Done Done Sent false
10501 3678 Acked Read 1 Done Done Sent false
10502 3679 Acked Read 2 Done Done Sent false
10503 3680 Acked Write 3 Done Done Sent false
10504 3681 Acked Read 8 Done Done Sent false
10505 3682 Acked Read 4 Done Done Sent false
10506 3683 Acked Read 2 Done Done Sent false
10507 3684 Acked Write 5 Done Done Sent false
10508 3685 Acked Flush 0 Done Done Sent false
10509 3686 Acked Flush 0 Done Done Sent false
10510 3687 Acked Read 1 Done Done Sent false
10511 3688 Acked Read 7 Done Done Sent false
10512 3689 Acked Read 10 Done Done Sent false
10513 3690 Acked Read 7 Done Done Sent false
10514 3691 Acked Flush 0 Done Done Sent false
10515 3692 Acked Read 9 Done Done Sent false
10516 3693 Acked Write 9 Done Done Sent false
10517 3694 Acked Read 3 Done Done Sent false
10518 3695 Acked Write 4 Done Done Sent false
10519 3696 Acked Read 1 Done Done Sent false
10520 3697 Acked Write 3 Done Done Sent false
10521 3698 Acked Read 7 Done Done Sent false
10522 3699 Acked Write 1 Done Done Sent false
10523 3700 Acked Read 3 Done Done Sent false
10524 3701 Acked Flush 0 Done Done Sent false
10525 3702 Acked Read 8 Done Done Sent false
10526 3703 Acked Write 2 Done Done Sent false
10527 3704 Acked Read 10 Done Done Sent false
10528 3705 Acked Flush 0 Done Done Sent false
10529 3706 Acked Flush 0 Done Done Sent false
10530 3707 Acked Read 3 Done Done Sent false
10531 3708 Acked Read 7 Done Done Sent false
10532 3709 Acked Read 10 Done Done Sent false
10533 3710 Acked Read 5 Done Done Sent false
10534 3711 Acked Read 2 Done Done Sent false
10535 3712 Acked Read 7 Done Done Sent false
10536 3713 Acked Flush 0 Done Done Sent false
10537 3714 Acked Read 2 Done Done Sent false
10538 3715 Acked Read 9 Done Done Sent false
10539 3716 Acked Write 7 Done Done Sent false
10540 3717 Acked Flush 0 Done Done Sent false
10541 3718 Acked Read 5 Done Done Sent false
10542 3719 Acked Write 9 Done Done Sent false
10543 3720 Acked Write 10 Sent Sent Sent false
10544 STATES DS:0 DS:1 DS:2 TOTAL
10545 Sent 1 1 301 303
10546 Done 300 300 0 600
10547 Skipped 0 0 0 0
10548 Error 0 0 0 0
10549 Last Flush: 3717 3717 3419
10550 Downstairs last five completed: 3419 3418 3417 3416 3415
10551 Upstairs last five completed: 3720 3719 3718 3717 3716
10552 CLIENT: Up:0 ds:301 act:2
10553 [8830] got start action
10554 [8830] I am going to restart
10555 Starting downstairs at port 8830
10556 Make output file at "/tmp/dsc/downstairs-8830.txt"
10557 [8830][2] reports Starting
10558 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1208)
10559 [8830][2] reports Running
10560 ----------------------------------------------------------------
10561 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10562 States: Active Active Offline
10563 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10564 3420 Acked Read 10 Done Done Sent false
10565 3421 Acked Write 9 Done Done Sent false
10566 3422 Acked Read 8 Done Done Sent false
10567 3423 Acked Write 2 Done Done Sent false
10568 3424 Acked Read 2 Done Done Sent false
10569 3425 Acked Write 4 Done Done Sent false
10570 3426 Acked Write 9 Done Done Sent false
10571 3427 Acked Write 6 Done Done Sent false
10572 3428 Acked Write 10 Done Done Sent false
10573 3429 Acked Write 3 Done Done Sent false
10574 3430 Acked Read 7 Done Done Sent false
10575 3431 Acked Read 5 Done Done Sent false
10576 3432 Acked Write 3 Done Done Sent false
10577 3433 Acked Write 9 Done Done Sent false
10578 3434 Acked Read 9 Done Done Sent false
10579 3435 Acked Write 8 Done Done Sent false
10580 3436 Acked Write 7 Done Done Sent false
10581 3437 Acked Read 5 Done Done Sent false
10582 3438 Acked Read 2 Done Done Sent false
10583 3439 Acked Read 1 Done Done Sent false
10584 3440 Acked Read 6 Done Done Sent false
10585 3441 Acked Read 2 Done Done Sent false
10586 3442 Acked Read 8 Done Done Sent false
10587 3443 Acked Read 6 Done Done Sent false
10588 3444 Acked Flush 0 Done Done Sent false
10589 3445 Acked Write 3 Done Done Sent false
10590 3446 Acked Read 2 Done Done Sent false
10591 3447 Acked Write 4 Done Done Sent false
10592 3448 Acked Read 5 Done Done Sent false
10593 3449 Acked Flush 0 Done Done Sent false
10594 3450 Acked Read 2 Done Done Sent false
10595 3451 Acked Write 4 Done Done Sent false
10596 3452 Acked Write 6 Done Done Sent false
10597 3453 Acked Write 5 Done Done Sent false
10598 3454 Acked Write 3 Done Done Sent false
10599 3455 Acked Write 2 Done Done Sent false
10600 3456 Acked Write 5 Done Done Sent false
10601 3457 Acked Write 4 Done Done Sent false
10602 3458 Acked Read 9 Done Done Sent false
10603 3459 Acked Read 5 Done Done Sent false
10604 3460 Acked Read 4 Done Done Sent false
10605 3461 Acked Read 9 Done Done Sent false
10606 3462 Acked Write 1 Done Done Sent false
10607 3463 Acked Read 7 Done Done Sent false
10608 3464 Acked Write 1 Done Done Sent false
10609 3465 Acked Write 2 Done Done Sent false
10610 3466 Acked Write 1 Done Done Sent false
10611 3467 Acked Read 6 Done Done Sent false
10612 3468 Acked Read 2 Done Done Sent false
10613 3469 Acked Write 6 Done Done Sent false
10614 3470 Acked Read 8 Done Done Sent false
10615 3471 Acked Write 1 Done Done Sent false
10616 3472 Acked Write 9 Done Done Sent false
10617 3473 Acked Read 5 Done Done Sent false
10618 3474 Acked Write 7 Done Done Sent false
10619 3475 Acked Write 3 Done Done Sent false
10620 3476 Acked Read 7 Done Done Sent false
10621 3477 Acked Read 6 Done Done Sent false
10622 3478 Acked Write 8 Done Done Sent false
10623 3479 Acked Flush 0 Done Done Sent false
10624 3480 Acked Write 5 Done Done Sent false
10625 3481 Acked Read 4 Done Done Sent false
10626 3482 Acked Read 4 Done Done Sent false
10627 3483 Acked Flush 0 Done Done Sent false
10628 3484 Acked Read 5 Done Done Sent false
10629 3485 Acked Read 10 Done Done Sent false
10630 3486 Acked Read 8 Done Done Sent false
10631 3487 Acked Read 8 Done Done Sent false
10632 3488 Acked Read 1 Done Done Sent false
10633 3489 Acked Read 8 Done Done Sent false
10634 3490 Acked Write 4 Done Done Sent false
10635 3491 Acked Read 1 Done Done Sent false
10636 3492 Acked Write 7 Done Done Sent false
10637 3493 Acked Read 2 Done Done Sent false
10638 3494 Acked Read 7 Done Done Sent false
10639 3495 Acked Read 4 Done Done Sent false
10640 3496 Acked Read 5 Done Done Sent false
10641 3497 Acked Read 3 Done Done Sent false
10642 3498 Acked Write 9 Done Done Sent false
10643 3499 Acked Write 5 Done Done Sent false
10644 3500 Acked Read 1 Done Done Sent false
10645 3501 Acked Write 8 Done Done Sent false
10646 3502 Acked Read 3 Done Done Sent false
10647 3503 Acked Read 1 Done Done Sent false
10648 3504 Acked Write 8 Done Done Sent false
10649 3505 Acked Flush 0 Done Done Sent false
10650 3506 Acked Read 6 Done Done Sent false
10651 3507 Acked Flush 0 Done Done Sent false
10652 3508 Acked Read 10 Done Done Sent false
10653 3509 Acked Read 9 Done Done Sent false
10654 3510 Acked Write 6 Done Done Sent false
10655 3511 Acked Read 1 Done Done Sent false
10656 3512 Acked Write 1 Done Done Sent false
10657 3513 Acked Read 5 Done Done Sent false
10658 3514 Acked Read 3 Done Done Sent false
10659 3515 Acked Read 2 Done Done Sent false
10660 3516 Acked Read 1 Done Done Sent false
10661 3517 Acked Read 6 Done Done Sent false
10662 3518 Acked Read 4 Done Done Sent false
10663 3519 Acked Read 10 Done Done Sent false
10664 3520 Acked Read 8 Done Done Sent false
10665 3521 Acked Write 2 Done Done Sent false
10666 3522 Acked Read 5 Done Done Sent false
10667 3523 Acked Read 1 Done Done Sent false
10668 3524 Acked Write 6 Done Done Sent false
10669 3525 Acked Write 10 Done Done Sent false
10670 3526 Acked Read 7 Done Done Sent false
10671 3527 Acked Flush 0 Done Done Sent false
10672 3528 Acked Read 3 Done Done Sent false
10673 3529 Acked Write 8 Done Done Sent false
10674 3530 Acked Write 2 Done Done Sent false
10675 3531 Acked Write 5 Done Done Sent false
10676 3532 Acked Write 10 Done Done Sent false
10677 3533 Acked Write 10 Done Done Sent false
10678 3534 Acked Write 10 Done Done Sent false
10679 3535 Acked Read 6 Done Done Sent false
10680 3536 Acked Read 8 Done Done Sent false
10681 3537 Acked Write 5 Done Done Sent false
10682 3538 Acked Read 7 Done Done Sent false
10683 3539 Acked Read 10 Done Done Sent false
10684 3540 Acked Write 7 Done Done Sent false
10685 3541 Acked Flush 0 Done Done Sent false
10686 3542 Acked Read 7 Done Done Sent false
10687 3543 Acked Read 3 Done Done Sent false
10688 3544 Acked Write 6 Done Done Sent false
10689 3545 Acked Read 2 Done Done Sent false
10690 3546 Acked Write 10 Done Done Sent false
10691 3547 Acked Read 1 Done Done Sent false
10692 3548 Acked Read 5 Done Done Sent false
10693 3549 Acked Write 4 Done Done Sent false
10694 3550 Acked Write 8 Done Done Sent false
10695 3551 Acked Read 3 Done Done Sent false
10696 3552 Acked Read 8 Done Done Sent false
10697 3553 Acked Read 7 Done Done Sent false
10698 3554 Acked Write 5 Done Done Sent false
10699 3555 Acked Read 4 Done Done Sent false
10700 3556 Acked Write 3 Done Done Sent false
10701 3557 Acked Write 1 Done Done Sent false
10702 3558 Acked Read 6 Done Done Sent false
10703 3559 Acked Flush 0 Done Done Sent false
10704 3560 Acked Write 4 Done Done Sent false
10705 3561 Acked Read 2 Done Done Sent false
10706 3562 Acked Read 7 Done Done Sent false
10707 3563 Acked Write 1 Done Done Sent false
10708 3564 Acked Read 6 Done Done Sent false
10709 3565 Acked Read 7 Done Done Sent false
10710 3566 Acked Read 1 Done Done Sent false
10711 3567 Acked Flush 0 Done Done Sent false
10712 3568 Acked Write 9 Done Done Sent false
10713 3569 Acked Write 9 Done Done Sent false
10714 3570 Acked Write 5 Done Done Sent false
10715 3571 Acked Read 8 Done Done Sent false
10716 3572 Acked Write 8 Done Done Sent false
10717 3573 Acked Write 5 Done Done Sent false
10718 3574 Acked Write 2 Done Done Sent false
10719 3575 Acked Read 8 Done Done Sent false
10720 3576 Acked Read 3 Done Done Sent false
10721 3577 Acked Write 5 Done Done Sent false
10722 3578 Acked Write 1 Done Done Sent false
10723 3579 Acked Read 9 Done Done Sent false
10724 3580 Acked Read 5 Done Done Sent false
10725 3581 Acked Flush 0 Done Done Sent false
10726 3582 Acked Read 9 Done Done Sent false
10727 3583 Acked Read 6 Done Done Sent false
10728 3584 Acked Read 8 Done Done Sent false
10729 3585 Acked Read 7 Done Done Sent false
10730 3586 Acked Read 3 Done Done Sent false
10731 3587 Acked Read 3 Done Done Sent false
10732 3588 Acked Read 4 Done Done Sent false
10733 3589 Acked Write 2 Done Done Sent false
10734 3590 Acked Write 2 Done Done Sent false
10735 3591 Acked Read 3 Done Done Sent false
10736 3592 Acked Write 6 Done Done Sent false
10737 3593 Acked Read 10 Done Done Sent false
10738 3594 Acked Write 4 Done Done Sent false
10739 3595 Acked Write 1 Done Done Sent false
10740 3596 Acked Read 5 Done Done Sent false
10741 3597 Acked Flush 0 Done Done Sent false
10742 3598 Acked Write 8 Done Done Sent false
10743 3599 Acked Write 8 Done Done Sent false
10744 3600 Acked Read 3 Done Done Sent false
10745 3601 Acked Write 9 Done Done Sent false
10746 3602 Acked Write 5 Done Done Sent false
10747 3603 Acked Flush 0 Done Done Sent false
10748 3604 Acked Write 6 Done Done Sent false
10749 3605 Acked Read 3 Done Done Sent false
10750 3606 Acked Flush 0 Done Done Sent false
10751 3607 Acked Write 8 Done Done Sent false
10752 3608 Acked Read 3 Done Done Sent false
10753 3609 Acked Write 5 Done Done Sent false
10754 3610 Acked Read 1 Done Done Sent false
10755 3611 Acked Write 6 Done Done Sent false
10756 3612 Acked Read 3 Done Done Sent false
10757 3613 Acked Flush 0 Done Done Sent false
10758 3614 Acked Read 3 Done Done Sent false
10759 3615 Acked Read 8 Done Done Sent false
10760 3616 Acked Flush 0 Done Done Sent false
10761 3617 Acked Flush 0 Done Done Sent false
10762 3618 Acked Flush 0 Done Done Sent false
10763 3619 Acked Read 1 Done Done Sent false
10764 3620 Acked Write 6 Done Done Sent false
10765 3621 Acked Read 1 Done Done Sent false
10766 3622 Acked Read 6 Done Done Sent false
10767 3623 Acked Read 8 Done Done Sent false
10768 3624 Acked Write 8 Done Done Sent false
10769 3625 Acked Read 10 Done Done Sent false
10770 3626 Acked Read 5 Done Done Sent false
10771 3627 Acked Read 7 Done Done Sent false
10772 3628 Acked Read 5 Done Done Sent false
10773 3629 Acked Read 8 Done Done Sent false
10774 3630 Acked Write 10 Done Done Sent false
10775 3631 Acked Read 6 Done Done Sent false
10776 3632 Acked Read 2 Done Done Sent false
10777 3633 Acked Read 9 Done Done Sent false
10778 3634 Acked Write 4 Done Done Sent false
10779 3635 Acked Read 1 Done Done Sent false
10780 3636 Acked Write 6 Done Done Sent false
10781 3637 Acked Read 5 Done Done Sent false
10782 3638 Acked Write 4 Done Done Sent false
10783 3639 Acked Write 10 Done Done Sent false
10784 3640 Acked Read 9 Done Done Sent false
10785 3641 Acked Flush 0 Done Done Sent false
10786 3642 Acked Flush 0 Done Done Sent false
10787 3643 Acked Flush 0 Done Done Sent false
10788 3644 Acked Read 4 Done Done Sent false
10789 3645 Acked Write 7 Done Done Sent false
10790 3646 Acked Flush 0 Done Done Sent false
10791 3647 Acked Write 2 Done Done Sent false
10792 3648 Acked Read 10 Done Done Sent false
10793 3649 Acked Read 2 Done Done Sent false
10794 3650 Acked Read 4 Done Done Sent false
10795 3651 Acked Read 8 Done Done Sent false
10796 3652 Acked Flush 0 Done Done Sent false
10797 3653 Acked Write 4 Done Done Sent false
10798 3654 Acked Write 9 Done Done Sent false
10799 3655 Acked Write 2 Done Done Sent false
10800 3656 Acked Read 10 Done Done Sent false
10801 3657 Acked Write 6 Done Done Sent false
10802 3658 Acked Write 7 Done Done Sent false
10803 3659 Acked Write 2 Done Done Sent false
10804 3660 Acked Read 2 Done Done Sent false
10805 3661 Acked Write 6 Done Done Sent false
10806 3662 Acked Read 6 Done Done Sent false
10807 3663 Acked Read 5 Done Done Sent false
10808 3664 Acked Flush 0 Done Done Sent false
10809 3665 Acked Write 5 Done Done Sent false
10810 3666 Acked Read 8 Done Done Sent false
10811 3667 Acked Flush 0 Done Done Sent false
10812 3668 Acked Write 10 Done Done Sent false
10813 3669 Acked Write 1 Done Done Sent false
10814 3670 Acked Read 4 Done Done Sent false
10815 3671 Acked Read 5 Done Done Sent false
10816 3672 Acked Write 5 Done Done Sent false
10817 3673 Acked Write 5 Done Done Sent false
10818 3674 Acked Flush 0 Done Done Sent false
10819 3675 Acked Read 8 Done Done Sent false
10820 3676 Acked Read 9 Done Done Sent false
10821 3677 Acked Read 5 Done Done Sent false
10822 3678 Acked Read 1 Done Done Sent false
10823 3679 Acked Read 2 Done Done Sent false
10824 3680 Acked Write 3 Done Done Sent false
10825 3681 Acked Read 8 Done Done Sent false
10826 3682 Acked Read 4 Done Done Sent false
10827 3683 Acked Read 2 Done Done Sent false
10828 3684 Acked Write 5 Done Done Sent false
10829 3685 Acked Flush 0 Done Done Sent false
10830 3686 Acked Flush 0 Done Done Sent false
10831 3687 Acked Read 1 Done Done Sent false
10832 3688 Acked Read 7 Done Done Sent false
10833 3689 Acked Read 10 Done Done Sent false
10834 3690 Acked Read 7 Done Done Sent false
10835 3691 Acked Flush 0 Done Done Sent false
10836 3692 Acked Read 9 Done Done Sent false
10837 3693 Acked Write 9 Done Done Sent false
10838 3694 Acked Read 3 Done Done Sent false
10839 3695 Acked Write 4 Done Done Sent false
10840 3696 Acked Read 1 Done Done Sent false
10841 3697 Acked Write 3 Done Done Sent false
10842 3698 Acked Read 7 Done Done Sent false
10843 3699 Acked Write 1 Done Done Sent false
10844 3700 Acked Read 3 Done Done Sent false
10845 3701 Acked Flush 0 Done Done Sent false
10846 3702 Acked Read 8 Done Done Sent false
10847 3703 Acked Write 2 Done Done Sent false
10848 3704 Acked Read 10 Done Done Sent false
10849 3705 Acked Flush 0 Done Done Sent false
10850 3706 Acked Flush 0 Done Done Sent false
10851 3707 Acked Read 3 Done Done Sent false
10852 3708 Acked Read 7 Done Done Sent false
10853 3709 Acked Read 10 Done Done Sent false
10854 3710 Acked Read 5 Done Done Sent false
10855 3711 Acked Read 2 Done Done Sent false
10856 3712 Acked Read 7 Done Done Sent false
10857 3713 Acked Flush 0 Done Done Sent false
10858 3714 Acked Read 2 Done Done Sent false
10859 3715 Acked Read 9 Done Done Sent false
10860 3716 Acked Write 7 Done Done Sent false
10861 3717 Acked Flush 0 Done Done Sent false
10862 3718 Acked Read 5 Done Done Sent false
10863 3719 Acked Write 9 Done Done Sent false
10864 3720 Acked Write 10 Done Done Sent false
10865 3721 Acked Flush 0 Done Done Sent false
10866 STATES DS:0 DS:1 DS:2 TOTAL
10867 Sent 0 0 302 302
10868 Done 302 302 0 604
10869 Skipped 0 0 0 0
10870 Error 0 0 0 0
10871 Last Flush: 3721 3721 3419
10872 Downstairs last five completed: 3419 3418 3417 3416 3415
10873 Upstairs last five completed: 3721 3720 3719 3718 3717
10874 CLIENT: Up:0 ds:302 act:2
108752024-11-07T14:05:34.186ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108762024-11-07T14:05:34.186ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64402) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108772024-11-07T14:05:34.187ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108782024-11-07T14:05:34.188ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108792024-11-07T14:05:34.188ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108802024-11-07T14:05:34.188ZINFOcrucible: send last flush ID to this DS: 3419 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108812024-11-07T14:05:34.188ZINFOcrucible: Replied this last flush ID: 3419 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108822024-11-07T14:05:34.188ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108832024-11-07T14:05:34.188ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108842024-11-07T14:05:34.188ZINFOcrucible: [2] client re-new 302 jobs since flush 3419 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108852024-11-07T14:05:34.188ZINFOcrucible: [2] Replayed 302 jobs since flush: 3419 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
10886 ----------------------------------------------------------------
10887 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10888 Downstairs last five completed: 3721 3720 3719 3718 3717
10889 Upstairs last five completed: 3721 3720 3719 3718 3717
10890 CLIENT: Up:0 ds:0 act:3
10891 Replay: All jobs finished
10892 Main task has work to do, go find it
10893 got dsc Stop(2)
10894 stop 2
10895 Nov 07 14:05:36.467 INFO request completed, latency_us: 367, response_code: 204, uri: /stop/cid/2, method: POST, [8830] Got stop action so:false kr:false
10896 req_id: 1cbec49e-b0b8-4b5d-99ed-f5e9c1178530, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
108972024-11-07T14:05:36.468ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108982024-11-07T14:05:36.468ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
108992024-11-07T14:05:36.468ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
109002024-11-07T14:05:36.468ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
109012024-11-07T14:05:36.468ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
109022024-11-07T14:05:36.468ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
10903 Nov 07 14:05:36.468 INFO request completed, latency_us: 186, response_code: 200, uri: /state/cid/2, method: GET, req_id: b8755375-3093-48d6-ac3f-b31813137823, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
10904 [8830] Exited with: ExitStatus(unix_wait_status(9))
10905 [8830][2] reports Exit
10906 Nov 07 14:05:40.470 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/2, method: GET, req_id: f3e30b82-1805-427c-abe6-708408f30fcb, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
10907 001/300 Read block 775 len 16384
10908 002/300 Read block 728 len 16384
10909 003/300 Read block 2303 len 28672
10910 004/300 Write block 3451 len 12288 data: 3 2 2
10911 005/300 Write block 2318 len 4096 data: 2
10912 006/300 Write block 1306 len 36864 data: 2 2 2 2 2 2 2 2 2
10913 007/300 Write block 3028 len 32768 data: 3 3 3 3 3 3 3 3
10914 008/300 Write block 2640 len 28672 data: 2 3 3 3 2 2 2
10915 009/300 Read block 3126 len 40960
10916 010/300 Read block 2972 len 20480
10917 011/300 Write block 1809 len 8192 data: 4 4
10918 012/300 Read block 725 len 4096
10919 013/300 Write block 4148 len 12288 data: 4 4 4
10920 014/300 Write block 4438 len 32768 data: 4 5 4 4 3 3 3 3
10921 015/300 Flush
10922 016/300 Read block 3748 len 12288
10923 017/300 Read block 1255 len 20480
10924 018/300 Read block 1456 len 28672
10925 019/300 Read block 3191 len 20480
10926 020/300 Read block 3648 len 28672
10927 021/300 Read block 1299 len 36864
10928 022/300 Read block 42 len 36864
10929 023/300 Read block 1866 len 12288
10930 024/300 Read block 3438 len 40960
10931 025/300 Flush
10932 026/300 Read block 1418 len 24576
10933 027/300 Write block 3462 len 40960 data: 4 3 3 2 2 2 2 2 2 2
10934 028/300 Flush
10935 029/300 Read block 2029 len 12288
10936 030/300 Read block 2662 len 40960
10937 031/300 Read block 1604 len 16384
10938 032/300 Read block 4352 len 8192
10939 033/300 Write block 3702 len 12288 data: 4 5 5
10940 034/300 Write block 4702 len 4096 data: 4
10941 035/300 Read block 2796 len 32768
10942 036/300 Write block 981 len 36864 data: 4 3 4 4 3 3 3 3 3
10943 037/300 Write block 976 len 40960 data: 2 2 2 2 4 5 4 5 5 4
10944 038/300 Read block 1702 len 4096
10945 039/300 Read block 1596 len 16384
10946 040/300 Read block 884 len 16384
10947 041/300 Read block 2482 len 28672
10948 042/300 Write block 3623 len 12288 data: 5 4 3
10949 043/300 Read block 3456 len 4096
10950 044/300 Write block 3866 len 36864 data: 3 3 3 3 4 3 3 4 4
10951 045/300 Write block 1608 len 4096 data: 3
10952 046/300 Write block 262 len 12288 data: 2 2 2
10953 047/300 Write block 4755 len 16384 data: 6 5 5 5
10954 048/300 Write block 3273 len 36864 data: 5 4 4 3 3 4 4 4 3
10955 049/300 Read block 1980 len 12288
10956 050/300 Write block 2769 len 4096 data: 2
10957 051/300 Read block 2420 len 4096
10958 052/300 Read block 4788 len 40960
10959 053/300 Read block 299 len 36864
10960 054/300 Write block 3174 len 20480 data: 2 2 3 3 4
10961 055/300 Write block 3048 len 20480 data: 2 4 4 4 4
10962 056/300 Flush
10963 057/300 Read block 2901 len 8192
10964 058/300 Flush
10965 059/300 Read block 3465 len 28672
10966 060/300 Read block 4176 len 40960
10967 061/300 Read block 4523 len 20480
10968 062/300 Read block 638 len 36864
10969 063/300 Write block 3972 len 32768 data: 4 4 4 3 3 3 2 2
10970 064/300 Read block 3802 len 40960
10971 065/300 Write block 3114 len 28672 data: 3 4 4 3 3 3 3
10972 066/300 Read block 2840 len 20480
10973 067/300 Read block 3460 len 4096
10974 068/300 Read block 1798 len 24576
10975 069/300 Write block 1862 len 8192 data: 2 2
10976 070/300 Write block 4383 len 32768 data: 4 5 5 4 4 4 4 4
10977 071/300 Read block 1013 len 36864
10978 072/300 Read block 2312 len 28672
10979 073/300 Read block 1149 len 40960
10980 074/300 Flush
10981 075/300 Write block 3196 len 4096 data: 3
10982 076/300 Write block 1869 len 20480 data: 2 2 2 2 3
10983 077/300 Read block 1843 len 4096
10984 078/300 Read block 4605 len 12288
10985 079/300 Read block 4827 len 40960
10986 080/300 Flush
10987 081/300 Read block 583 len 12288
10988 082/300 Flush
10989 083/300 Read block 625 len 40960
10990 084/300 Write block 2146 len 4096 data: 2
10991 085/300 Write block 333 len 16384 data: 2 2 2 2
10992 086/300 Read block 1193 len 24576
10993 087/300 Write block 2264 len 32768 data: 2 2 2 2 3 3 4 4
10994 088/300 Read block 1298 len 40960
10995 089/300 Write block 3375 len 12288 data: 3 4 4
10996 090/300 Read block 2721 len 12288
10997 091/300 Flush
10998 092/300 Read block 3934 len 40960
10999 093/300 Read block 2001 len 28672
11000 094/300 Read block 4125 len 20480
11001 095/300 Read block 4897 len 36864
11002 096/300 Read block 2652 len 4096
11003 097/300 Read block 2040 len 16384
11004 098/300 Read block 267 len 40960
11005 099/300 Flush
11006 100/300 Read block 133 len 16384
11007 101/300 Flush
11008 102/300 Read block 3041 len 32768
11009 103/300 Write block 577 len 8192 data: 3 3
11010 104/300 Write block 1865 len 16384 data: 2 2 2 2
11011 105/300 Read block 1911 len 4096
11012 106/300 Read block 3011 len 8192
11013 107/300 Read block 525 len 36864
11014 108/300 Flush
11015 109/300 Write block 3692 len 12288 data: 4 4 4
11016 110/300 Read block 869 len 4096
11017 111/300 Flush
11018 112/300 Read block 1365 len 8192
11019 113/300 Write block 2879 len 40960 data: 2 3 3 4 4 4 4 3 3 3
11020 114/300 Read block 3771 len 4096
11021 115/300 Flush
11022 116/300 Read block 4307 len 4096
11023 117/300 Read block 854 len 32768
11024 118/300 Write block 877 len 32768 data: 4 4 3 2 2 2 3 3
11025 119/300 Read block 3978 len 32768
11026 120/300 Write block 1580 len 40960 data: 3 3 2 2 2 2 3 3 3 3
11027 121/300 Write block 2068 len 16384 data: 2 2 2 2
11028 122/300 Read block 3623 len 8192
11029 123/300 Write block 3292 len 28672 data: 4 3 3 3 3 3 3
11030 124/300 Write block 1597 len 40960 data: 4 4 4 5 5 5 5 5 4 3
11031 125/300 Flush
11032 126/300 Read block 3271 len 28672
11033 127/300 Read block 3771 len 28672
11034 128/300 Read block 2384 len 12288
11035 129/300 Read block 3890 len 32768
11036 130/300 Flush
11037 131/300 Read block 1194 len 8192
11038 132/300 Read block 1297 len 20480
11039 133/300 Flush
11040 134/300 Read block 1731 len 36864
11041 135/300 Write block 4143 len 16384 data: 2 2 2 3
11042 136/300 Write block 4794 len 16384 data: 2 2 3 3
11043 137/300 Write block 3635 len 4096 data: 3
11044 138/300 Write block 3924 len 24576 data: 5 4 4 4 4 2
11045 139/300 Read block 4118 len 32768
11046 140/300 Read block 1667 len 36864
11047 141/300 Read block 1519 len 12288
11048 142/300 Flush
11049 143/300 Read block 1969 len 40960
11050 144/300 Read block 4061 len 40960
11051 145/300 Read block 3981 len 8192
11052 146/300 Write block 3432 len 8192 data: 3 3
11053 147/300 Read block 3511 len 24576
11054 148/300 Read block 2450 len 8192
11055 149/300 Write block 4508 len 28672 data: 4 4 4 4 4 4 4
11056 150/300 Write block 4945 len 24576 data: 2 2 2 2 3 4
11057 151/300 Read block 4372 len 12288
11058 152/300 Read block 4865 len 20480
11059 153/300 Read block 3518 len 24576
11060 154/300 Write block 1771 len 20480 data: 4 3 3 3 3
11061 155/300 Flush
11062 156/300 Write block 335 len 24576 data: 3 3 2 2 2 2
11063 157/300 Read block 1607 len 36864
11064 158/300 Read block 528 len 32768
11065 159/300 Read block 466 len 32768
11066 160/300 Read block 4545 len 36864
11067 161/300 Read block 84 len 8192
11068 162/300 Write block 1353 len 20480 data: 3 5 4 6 6
11069 163/300 Write block 1827 len 28672 data: 2 3 3 3 3 3 2
11070 164/300 Write block 3837 len 12288 data: 3 3 3
11071 165/300 Write block 1874 len 36864 data: 3 4 4 5 5 5 3 3 3
11072 166/300 Flush
11073 167/300 Read block 4082 len 24576
11074 168/300 Read block 3182 len 12288
11075 169/300 Flush
11076 170/300 Write block 3828 len 8192 data: 5 4
11077 171/300 Read block 3902 len 20480
11078 172/300 Read block 2847 len 32768
11079 173/300 Write block 4391 len 40960 data: 4 3 3 2 2 2 2 2 2 2
11080 174/300 Write block 82 len 32768 data: 3 3 3 3 3 3 4 4
11081 175/300 Read block 908 len 20480
11082 176/300 Read block 2704 len 16384
11083 177/300 Read block 2792 len 28672
11084 178/300 Flush
11085 179/300 Write block 4697 len 4096 data: 5
11086 180/300 Read block 2521 len 8192
11087 181/300 Read block 2039 len 4096
11088 182/300 Read block 4195 len 32768
11089 183/300 Read block 3298 len 16384
11090 184/300 Write block 2658 len 8192 data: 4 4
11091 185/300 Read block 2983 len 12288
11092 186/300 Flush
11093 187/300 Read block 2921 len 4096
11094 188/300 Write block 2478 len 40960 data: 4 4 4 4 4 4 3 4 3 3
11095 189/300 Read block 1738 len 4096
11096 190/300 Read block 1773 len 36864
11097 191/300 Read block 3596 len 36864
11098 192/300 Write block 1004 len 12288 data: 2 2 2
11099 193/300 Write block 2668 len 40960 data: 2 2 2 2 2 2 2 3 3 3
11100 194/300 Read block 4544 len 24576
11101 195/300 Flush
11102 196/300 Read block 1463 len 32768
11103 197/300 Write block 3927 len 8192 data: 5 5
11104 198/300 Flush
11105 199/300 Read block 3101 len 24576
11106 200/300 Read block 613 len 28672
11107 201/300 Read block 2888 len 20480
11108 202/300 Read block 4932 len 24576
11109 203/300 Write block 4119 len 36864 data: 4 3 3 3 4 4 3 3 3
11110 204/300 Read block 1809 len 28672
11111 205/300 Write block 236 len 36864 data: 3 4 4 3 4 3 3 3 3
11112 206/300 Read block 2926 len 24576
11113 207/300 Read block 1694 len 36864
11114 208/300 Write block 10 len 20480 data: 2 2 3 3 3
11115 209/300 Read block 2075 len 8192
11116 210/300 Write block 3173 len 40960 data: 3 3 3 4 4 5 3 2 2 2
11117 211/300 Read block 3904 len 36864
11118 212/300 Write block 2482 len 8192 data: 5 5
11119 213/300 Read block 3689 len 28672
11120 214/300 Read block 1021 len 40960
11121 215/300 Write block 2491 len 4096 data: 3
11122 216/300 Read block 3872 len 24576
11123 217/300 Read block 3976 len 8192
11124 218/300 Write block 229 len 28672 data: 2 2 2 2 2 2 2
11125 219/300 Read block 851 len 16384
11126 220/300 Read block 3780 len 20480
11127 221/300 Read block 851 len 16384
11128 222/300 Write block 1036 len 40960 data: 2 2 2 3 3 4 4 5 5 5
11129 223/300 Read block 1309 len 28672
11130 224/300 Read block 738 len 24576
11131 225/300 Write block 4870 len 4096 data: 3
11132 226/300 Read block 3176 len 20480
11133 227/300 Read block 3155 len 36864
11134 228/300 Write block 325 len 24576 data: 3 2 2 2 2 2
11135 229/300 Write block 4787 len 24576 data: 2 3 3 3 3 3
11136 230/300 Read block 4859 len 36864
11137 231/300 Read block 700 len 20480
11138 232/300 Read block 4898 len 20480
11139 233/300 Read block 4653 len 4096
11140 234/300 Read block 79 len 40960
11141 235/300 Read block 1980 len 12288
11142 236/300 Read block 295 len 36864
11143 237/300 Flush
11144 238/300 Flush
11145 239/300 Read block 591 len 28672
11146 240/300 Read block 4608 len 12288
11147 241/300 Flush
11148 242/300 Write block 256 len 32768 data: 3 2 2 2 2 2 3 3
11149 243/300 Write block 3152 len 40960 data: 3 3 4 4 4 3 3 3 3 2
11150 244/300 Read block 3351 len 24576
11151 245/300 Flush
11152 246/300 Read block 1919 len 8192
11153 247/300 Flush
11154 248/300 Read block 3549 len 12288
11155 249/300 Read block 328 len 16384
11156 250/300 Read block 3482 len 16384
11157 251/300 Read block 32 len 20480
11158 252/300 Write block 2786 len 28672 data: 4 3 3 3 3 3 3
11159 253/300 Write block 1436 len 12288 data: 2 3 3
11160 254/300 Write block 3345 len 24576 data: 4 3 2 2 2 2
11161 255/300 Read block 571 len 8192
11162 256/300 Write block 4204 len 4096 data: 4
11163 257/300 Write block 1978 len 20480 data: 3 3 3 4 4
11164 258/300 Read block 1453 len 12288
11165 259/300 Read block 567 len 8192
11166 260/300 Read block 4234 len 32768
11167 261/300 Write block 1667 len 28672 data: 7 5 5 5 3 3 4
11168 262/300 Write block 1114 len 12288 data: 2 2 2
11169 263/300 Write block 4672 len 36864 data: 3 3 3 3 3 3 3 3 2
11170 264/300 Flush
11171 265/300 Read block 490 len 40960
11172 266/300 Write block 2539 len 12288 data: 5 5 5
11173 267/300 Write block 2950 len 24576 data: 2 2 2 2 2 2
11174 268/300 Write block 1332 len 24576 data: 4 4 3 3 3 3
11175 269/300 Write block 3305 len 36864 data: 2 2 2 2 2 2 2 2 2
11176 270/300 Read block 1215 len 16384
11177 271/300 Flush
11178 272/300 Flush
11179 273/300 Flush
11180 274/300 Read block 3635 len 8192
11181 275/300 Read block 3311 len 4096
11182 276/300 Read block 3956 len 28672
11183 277/300 Read block 2658 len 32768
11184 278/300 Read block 325 len 20480
11185 279/300 Write block 4147 len 24576 data: 3 5 5 5 4 3
11186 280/300 Write block 2691 len 16384 data: 3 3 3 4
11187 281/300 Write block 4995 len 4096 data: 2
11188 282/300 Write block 447 len 24576 data: 2 2 2 4 4 6
11189 283/300 Write block 3987 len 28672 data: 2 3 3 2 4 3 3
11190 284/300 Read block 2560 len 24576
11191 285/300 Read block 617 len 24576
11192 286/300 Write block 2244 len 36864 data: 2 2 2 2 2 2 2 2 2
11193 287/300 Write block 2033 len 8192 data: 3 2
11194 288/300 Write block 400 len 24576 data: 3 3 4 5 5 5
11195 289/300 Write block 3710 len 8192 data: 3 4
11196 290/300 Write block 48 len 24576 data: 3 3 3 3 3 3
11197 291/300 Read block 997 len 32768
11198 292/300 Read block 4748 len 32768
11199 293/300 Read block 71 len 40960
11200 294/300 Read block 2039 len 32768
11201 295/300 Write block 2132 len 12288 data: 3 3 3
11202 296/300 Write block 13 len 24576 data: 4 4 3 3 3 3
11203 297/300 Write block 563 len 28672 data: 3 3 2 3 4 4 4
11204 298/300 Write block 1648 len 28672 data: 6 5 5 5 5 5 5
11205 299/300 Write block 3181 len 20480 data: 3 3 2 2 2
11206 300/300 Read block 730 len 4096
11207 301/300 Write block 2578 len 8192 data: 4 4
11208 Main task has work to do, go find it
11209 got dsc Start(2)
11210 start 2
11211 Nov 07 14:05:40.753 INFO request completed, latency_us: 196, response_code: 204, uri: /start/cid/2, method: POST, req_id: d47ba038-261e-4e08-9d82-16892bf064ae, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
11212 Replay: started 2, returned:Ok(())
11213 ----------------------------------------------------------------
11214 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11215 States: Active Active Offline
11216 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11217 3722 Acked Read 4 Done Done Sent false
11218 3723 Acked Read 4 Done Done Sent false
11219 3724 Acked Read 7 Done Done Sent false
11220 3725 Acked Write 3 Done Done Sent false
11221 3726 Acked Write 1 Done Done Sent false
11222 3727 Acked Write 9 Done Done Sent false
11223 3728 Acked Write 8 Done Done Sent false
11224 3729 Acked Write 7 Done Done Sent false
11225 3730 Acked Read 10 Done Done Sent false
11226 3731 Acked Read 5 Done Done Sent false
11227 3732 Acked Write 2 Done Done Sent false
11228 3733 Acked Read 1 Done Done Sent false
11229 3734 Acked Write 3 Done Done Sent false
11230 3735 Acked Write 8 Done Done Sent false
11231 3736 Acked Flush 0 Done Done Sent false
11232 3737 Acked Read 3 Done Done Sent false
11233 3738 Acked Read 5 Done Done Sent false
11234 3739 Acked Read 7 Done Done Sent false
11235 3740 Acked Read 5 Done Done Sent false
11236 3741 Acked Read 7 Done Done Sent false
11237 3742 Acked Read 9 Done Done Sent false
11238 3743 Acked Read 9 Done Done Sent false
11239 3744 Acked Read 3 Done Done Sent false
11240 3745 Acked Read 10 Done Done Sent false
11241 3746 Acked Flush 0 Done Done Sent false
11242 3747 Acked Read 6 Done Done Sent false
11243 3748 Acked Write 10 Done Done Sent false
11244 3749 Acked Flush 0 Done Done Sent false
11245 3750 Acked Read 3 Done Done Sent false
11246 3751 Acked Read 10 Done Done Sent false
11247 3752 Acked Read 4 Done Done Sent false
11248 3753 Acked Read 2 Done Done Sent false
11249 3754 Acked Write 3 Done Done Sent false
11250 3755 Acked Write 1 Done Done Sent false
11251 3756 Acked Read 8 Done Done Sent false
11252 3757 Acked Write 9 Done Done Sent false
11253 3758 Acked Write 10 Done Done Sent false
11254 3759 Acked Read 1 Done Done Sent false
11255 3760 Acked Read 4 Done Done Sent false
11256 3761 Acked Read 4 Done Done Sent false
11257 3762 Acked Read 7 Done Done Sent false
11258 3763 Acked Write 3 Done Done Sent false
11259 3764 Acked Read 1 Done Done Sent false
11260 3765 Acked Write 9 Done Done Sent false
11261 3766 Acked Write 1 Done Done Sent false
11262 3767 Acked Write 3 Done Done Sent false
11263 3768 Acked Write 4 Done Done Sent false
11264 3769 Acked Write 9 Done Done Sent false
11265 3770 Acked Read 3 Done Done Sent false
11266 3771 Acked Write 1 Done Done Sent false
11267 3772 Acked Read 1 Done Done Sent false
11268 3773 Acked Read 10 Done Done Sent false
11269 3774 Acked Read 9 Done Done Sent false
11270 3775 Acked Flush 0 Done Done Sent false
11271 3776 Acked Write 5 Done Done Sent false
11272 3777 Acked Write 5 Done Done Sent false
11273 3778 Acked Flush 0 Done Done Sent false
11274 3779 Acked Read 2 Done Done Sent false
11275 3780 Acked Flush 0 Done Done Sent false
11276 3781 Acked Read 7 Done Done Sent false
11277 3782 Acked Read 10 Done Done Sent false
11278 3783 Acked Read 5 Done Done Sent false
11279 3784 Acked Read 9 Done Done Sent false
11280 3785 Acked Write 8 Done Done Sent false
11281 3786 Acked Read 10 Done Done Sent false
11282 3787 Acked Write 7 Done Done Sent false
11283 3788 Acked Read 5 Done Done Sent false
11284 3789 Acked Read 1 Done Done Sent false
11285 3790 Acked Read 6 Done Done Sent false
11286 3791 Acked Write 2 Done Done Sent false
11287 3792 Acked Write 8 Done Done Sent false
11288 3793 Acked Read 9 Done Done Sent false
11289 3794 Acked Read 7 Done Done Sent false
11290 3795 Acked Read 10 Done Done Sent false
11291 3796 Acked Flush 0 Done Done Sent false
11292 3797 Acked Write 1 Done Done Sent false
11293 3798 Acked Write 5 Done Done Sent false
11294 3799 Acked Read 1 Done Done Sent false
11295 3800 Acked Read 3 Done Done Sent false
11296 3801 Acked Read 10 Done Done Sent false
11297 3802 Acked Flush 0 Done Done Sent false
11298 3803 Acked Read 3 Done Done Sent false
11299 3804 Acked Flush 0 Done Done Sent false
11300 3805 Acked Read 10 Done Done Sent false
11301 3806 Acked Write 1 Done Done Sent false
11302 3807 Acked Write 4 Done Done Sent false
11303 3808 Acked Read 6 Done Done Sent false
11304 3809 Acked Write 8 Done Done Sent false
11305 3810 Acked Read 10 Done Done Sent false
11306 3811 Acked Write 3 Done Done Sent false
11307 3812 Acked Read 3 Done Done Sent false
11308 3813 Acked Flush 0 Done Done Sent false
11309 3814 Acked Read 10 Done Done Sent false
11310 3815 Acked Read 7 Done Done Sent false
11311 3816 Acked Read 5 Done Done Sent false
11312 3817 Acked Read 9 Done Done Sent false
11313 3818 Acked Read 1 Done Done Sent false
11314 3819 Acked Read 4 Done Done Sent false
11315 3820 Acked Read 10 Done Done Sent false
11316 3821 Acked Flush 0 Done Done Sent false
11317 3822 Acked Read 4 Done Done Sent false
11318 3823 Acked Flush 0 Done Done Sent false
11319 3824 Acked Read 8 Done Done Sent false
11320 3825 Acked Write 2 Done Done Sent false
11321 3826 Acked Write 4 Done Done Sent false
11322 3827 Acked Read 1 Done Done Sent false
11323 3828 Acked Read 2 Done Done Sent false
11324 3829 Acked Read 9 Done Done Sent false
11325 3830 Acked Flush 0 Done Done Sent false
11326 3831 Acked Write 3 Done Done Sent false
11327 3832 Acked Read 1 Done Done Sent false
11328 3833 Acked Flush 0 Done Done Sent false
11329 3834 Acked Read 2 Done Done Sent false
11330 3835 Acked Write 10 Done Done Sent false
11331 3836 Acked Read 1 Done Done Sent false
11332 3837 Acked Flush 0 Done Done Sent false
11333 3838 Acked Read 1 Done Done Sent false
11334 3839 Acked Read 8 Done Done Sent false
11335 3840 Acked Write 8 Done Done Sent false
11336 3841 Acked Read 8 Done Done Sent false
11337 3842 Acked Write 10 Done Done Sent false
11338 3843 Acked Write 4 Done Done Sent false
11339 3844 Acked Read 2 Done Done Sent false
11340 3845 Acked Write 7 Done Done Sent false
11341 3846 Acked Write 10 Done Done Sent false
11342 3847 Acked Flush 0 Done Done Sent false
11343 3848 Acked Read 7 Done Done Sent false
11344 3849 Acked Read 7 Done Done Sent false
11345 3850 Acked Read 3 Done Done Sent false
11346 3851 Acked Read 8 Done Done Sent false
11347 3852 Acked Flush 0 Done Done Sent false
11348 3853 Acked Read 2 Done Done Sent false
11349 3854 Acked Read 5 Done Done Sent false
11350 3855 Acked Flush 0 Done Done Sent false
11351 3856 Acked Read 9 Done Done Sent false
11352 3857 Acked Write 4 Done Done Sent false
11353 3858 Acked Write 4 Done Done Sent false
11354 3859 Acked Write 1 Done Done Sent false
11355 3860 Acked Write 6 Done Done Sent false
11356 3861 Acked Read 8 Done Done Sent false
11357 3862 Acked Read 9 Done Done Sent false
11358 3863 Acked Read 3 Done Done Sent false
11359 3864 Acked Flush 0 Done Done Sent false
11360 3865 Acked Read 10 Done Done Sent false
11361 3866 Acked Read 10 Done Done Sent false
11362 3867 Acked Read 2 Done Done Sent false
11363 3868 Acked Write 2 Done Done Sent false
11364 3869 Acked Read 6 Done Done Sent false
11365 3870 Acked Read 2 Done Done Sent false
11366 3871 Acked Write 7 Done Done Sent false
11367 3872 Acked Write 6 Done Done Sent false
11368 3873 Acked Read 3 Done Done Sent false
11369 3874 Acked Read 5 Done Done Sent false
11370 3875 Acked Read 6 Done Done Sent false
11371 3876 Acked Write 5 Done Done Sent false
11372 3877 Acked Flush 0 Done Done Sent false
11373 3878 Acked Write 6 Done Done Sent false
11374 3879 Acked Read 9 Done Done Sent false
11375 3880 Acked Read 8 Done Done Sent false
11376 3881 Acked Read 8 Done Done Sent false
11377 3882 Acked Read 9 Done Done Sent false
11378 3883 Acked Read 2 Done Done Sent false
11379 3884 Acked Write 5 Done Done Sent false
11380 3885 Acked Write 7 Done Done Sent false
11381 3886 Acked Write 3 Done Done Sent false
11382 3887 Acked Write 9 Done Done Sent false
11383 3888 Acked Flush 0 Done Done Sent false
11384 3889 Acked Read 6 Done Done Sent false
11385 3890 Acked Read 3 Done Done Sent false
11386 3891 Acked Flush 0 Done Done Sent false
11387 3892 Acked Write 2 Done Done Sent false
11388 3893 Acked Read 5 Done Done Sent false
11389 3894 Acked Read 8 Done Done Sent false
11390 3895 Acked Write 10 Done Done Sent false
11391 3896 Acked Write 8 Done Done Sent false
11392 3897 Acked Read 5 Done Done Sent false
11393 3898 Acked Read 4 Done Done Sent false
11394 3899 Acked Read 7 Done Done Sent false
11395 3900 Acked Flush 0 Done Done Sent false
11396 3901 Acked Write 1 Done Done Sent false
11397 3902 Acked Read 2 Done Done Sent false
11398 3903 Acked Read 1 Done Done Sent false
11399 3904 Acked Read 8 Done Done Sent false
11400 3905 Acked Read 4 Done Done Sent false
11401 3906 Acked Write 2 Done Done Sent false
11402 3907 Acked Read 3 Done Done Sent false
11403 3908 Acked Flush 0 Done Done Sent false
11404 3909 Acked Read 1 Done Done Sent false
11405 3910 Acked Write 10 Done Done Sent false
11406 3911 Acked Read 1 Done Done Sent false
11407 3912 Acked Read 9 Done Done Sent false
11408 3913 Acked Read 9 Done Done Sent false
11409 3914 Acked Write 3 Done Done Sent false
11410 3915 Acked Write 10 Done Done Sent false
11411 3916 Acked Read 6 Done Done Sent false
11412 3917 Acked Flush 0 Done Done Sent false
11413 3918 Acked Read 8 Done Done Sent false
11414 3919 Acked Write 2 Done Done Sent false
11415 3920 Acked Flush 0 Done Done Sent false
11416 3921 Acked Read 6 Done Done Sent false
11417 3922 Acked Read 7 Done Done Sent false
11418 3923 Acked Read 5 Done Done Sent false
11419 3924 Acked Read 6 Done Done Sent false
11420 3925 Acked Write 9 Done Done Sent false
11421 3926 Acked Read 7 Done Done Sent false
11422 3927 Acked Write 9 Done Done Sent false
11423 3928 Acked Read 6 Done Done Sent false
11424 3929 Acked Read 9 Done Done Sent false
11425 3930 Acked Write 5 Done Done Sent false
11426 3931 Acked Read 2 Done Done Sent false
11427 3932 Acked Write 10 Done Done Sent false
11428 3933 Acked Read 9 Done Done Sent false
11429 3934 Acked Write 2 Done Done Sent false
11430 3935 Acked Read 7 Done Done Sent false
11431 3936 Acked Read 10 Done Done Sent false
11432 3937 Acked Write 1 Done Done Sent false
11433 3938 Acked Read 6 Done Done Sent false
11434 3939 Acked Read 2 Done Done Sent false
11435 3940 Acked Write 7 Done Done Sent false
11436 3941 Acked Read 4 Done Done Sent false
11437 3942 Acked Read 5 Done Done Sent false
11438 3943 Acked Read 4 Done Done Sent false
11439 3944 Acked Write 10 Done Done Sent false
11440 3945 Acked Read 7 Done Done Sent false
11441 3946 Acked Read 6 Done Done Sent false
11442 3947 Acked Write 1 Done Done Sent false
11443 3948 Acked Read 5 Done Done Sent false
11444 3949 Acked Read 9 Done Done Sent false
11445 3950 Acked Write 6 Done Done Sent false
11446 3951 Acked Write 6 Done Done Sent false
11447 3952 Acked Read 9 Done Done Sent false
11448 3953 Acked Read 5 Done Done Sent false
11449 3954 Acked Read 5 Done Done Sent false
11450 3955 Acked Read 1 Done Done Sent false
11451 3956 Acked Read 10 Done Done Sent false
11452 3957 Acked Read 3 Done Done Sent false
11453 3958 Acked Read 9 Done Done Sent false
11454 3959 Acked Flush 0 Done Done Sent false
11455 3960 Acked Flush 0 Done Done Sent false
11456 3961 Acked Read 7 Done Done Sent false
11457 3962 Acked Read 3 Done Done Sent false
11458 3963 Acked Flush 0 Done Done Sent false
11459 3964 Acked Write 8 Done Done Sent false
11460 3965 Acked Write 10 Done Done Sent false
11461 3966 Acked Read 6 Done Done Sent false
11462 3967 Acked Flush 0 Done Done Sent false
11463 3968 Acked Read 2 Done Done Sent false
11464 3969 Acked Flush 0 Done Done Sent false
11465 3970 Acked Read 3 Done Done Sent false
11466 3971 Acked Read 4 Done Done Sent false
11467 3972 Acked Read 4 Done Done Sent false
11468 3973 Acked Read 5 Done Done Sent false
11469 3974 Acked Write 7 Done Done Sent false
11470 3975 Acked Write 3 Done Done Sent false
11471 3976 Acked Write 6 Done Done Sent false
11472 3977 Acked Read 2 Done Done Sent false
11473 3978 Acked Write 1 Done Done Sent false
11474 3979 Acked Write 5 Done Done Sent false
11475 3980 Acked Read 3 Done Done Sent false
11476 3981 Acked Read 2 Done Done Sent false
11477 3982 Acked Read 8 Done Done Sent false
11478 3983 Acked Write 7 Done Done Sent false
11479 3984 Acked Write 3 Done Done Sent false
11480 3985 Acked Write 9 Done Done Sent false
11481 3986 Acked Flush 0 Done Done Sent false
11482 3987 Acked Read 10 Done Done Sent false
11483 3988 Acked Write 3 Done Done Sent false
11484 3989 Acked Write 6 Done Done Sent false
11485 3990 Acked Write 6 Done Done Sent false
11486 3991 Acked Write 9 Done Done Sent false
11487 3992 Acked Read 4 Done Done Sent false
11488 3993 Acked Flush 0 Done Done Sent false
11489 3994 Acked Flush 0 Done Done Sent false
11490 3995 Acked Flush 0 Done Done Sent false
11491 3996 Acked Read 2 Done Done Sent false
11492 3997 Acked Read 1 Done Done Sent false
11493 3998 Acked Read 7 Done Done Sent false
11494 3999 Acked Read 8 Done Done Sent false
11495 4000 Acked Read 5 Done Done Sent false
11496 4001 Acked Write 6 Done Done Sent false
11497 4002 Acked Write 4 Done Done Sent false
11498 4003 Acked Write 1 Done Done Sent false
11499 4004 Acked Write 6 Done Done Sent false
11500 4005 Acked Write 7 Done Done Sent false
11501 4006 Acked Read 6 Done Done Sent false
11502 4007 Acked Read 6 Done Done Sent false
11503 4008 Acked Write 9 Done Done Sent false
11504 4009 Acked Write 2 Done Done Sent false
11505 4010 Acked Write 6 Done Done Sent false
11506 4011 Acked Write 2 Done Done Sent false
11507 4012 Acked Write 6 Done Done Sent false
11508 4013 Acked Read 8 Done Done Sent false
11509 4014 Acked Read 8 Done Done Sent false
11510 4015 Acked Read 10 Done Done Sent false
11511 4016 Acked Read 8 Done Done Sent false
11512 4017 Acked Write 3 Done Done Sent false
11513 4018 Acked Write 6 Done Done Sent false
11514 4019 Acked Write 7 Done Done Sent false
11515 4020 Acked Write 7 Done Done Sent false
11516 4021 Acked Write 5 Done Done Sent false
11517 4022 Acked Read 1 Done Done Sent false
11518 4023 Acked Write 2 Done Done Sent false
11519 STATES DS:0 DS:1 DS:2 TOTAL
11520 Sent 0 0 302 302
11521 Done 302 302 0 604
11522 Skipped 0 0 0 0
11523 Error 0 0 0 0
11524 Last Flush: 3995 3995 3721
11525 Downstairs last five completed: 3721 3720 3719 3718 3717
11526 Upstairs last five completed: 4023 4022 4021 4020 4019
11527 CLIENT: Up:0 ds:302 act:2
11528 ----------------------------------------------------------------
11529 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
11530 States: Active Active Offline
11531 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11532 3722 Acked Read 4 Done Done Sent false
11533 3723 Acked Read 4 Done Done Sent false
11534 3724 Acked Read 7 Done Done Sent false
11535 3725 Acked Write 3 Done Done Sent false
11536 3726 Acked Write 1 Done Done Sent false
11537 3727 Acked Write 9 Done Done Sent false
11538 3728 Acked Write 8 Done Done Sent false
11539 3729 Acked Write 7 Done Done Sent false
11540 3730 Acked Read 10 Done Done Sent false
11541 3731 Acked Read 5 Done Done Sent false
11542 3732 Acked Write 2 Done Done Sent false
11543 3733 Acked Read 1 Done Done Sent false
11544 3734 Acked Write 3 Done Done Sent false
11545 3735 Acked Write 8 Done Done Sent false
11546 3736 Acked Flush 0 Done Done Sent false
11547 3737 Acked Read 3 Done Done Sent false
11548 3738 Acked Read 5 Done Done Sent false
11549 3739 Acked Read 7 Done Done Sent false
11550 3740 Acked Read 5 Done Done Sent false
11551 3741 Acked Read 7 Done Done Sent false
11552 3742 Acked Read 9 Done Done Sent false
11553 3743 Acked Read 9 Done Done Sent false
11554 3744 Acked Read 3 Done Done Sent false
11555 3745 Acked Read 10 Done Done Sent false
11556 3746 Acked Flush 0 Done Done Sent false
11557 3747 Acked Read 6 Done Done Sent false
11558 3748 Acked Write 10 Done Done Sent false
11559 3749 Acked Flush 0 Done Done Sent false
11560 3750 Acked Read 3 Done Done Sent false
11561 3751 Acked Read 10 Done Done Sent false
11562 3752 Acked Read 4 Done Done Sent false
11563 3753 Acked Read 2 Done Done Sent false
11564 3754 Acked Write 3 Done Done Sent false
11565 3755 Acked Write 1 Done Done Sent false
11566 3756 Acked Read 8 Done Done Sent false
11567 3757 Acked Write 9 Done Done Sent false
11568 3758 Acked Write 10 Done Done Sent false
11569 3759 Acked Read 1 Done Done Sent false
11570 3760 Acked Read 4 Done Done Sent false
11571 3761 Acked Read 4 Done Done Sent false
11572 3762 Acked Read 7 Done Done Sent false
11573 3763 Acked Write 3 Done Done Sent false
11574 3764 Acked Read 1 Done Done Sent false
11575 3765 Acked Write 9 Done Done Sent false
11576 3766 Acked Write 1 Done Done Sent false
11577 3767 Acked Write 3 Done Done Sent false
11578 3768 Acked Write 4 Done Done Sent false
11579 3769 Acked Write 9 Done Done Sent false
11580 3770 Acked Read 3 Done Done Sent false
11581 3771 Acked Write 1 Done Done Sent false
11582 3772 Acked Read 1 Done Done Sent false
11583 3773 Acked Read 10 Done Done Sent false
11584 3774 Acked Read 9 Done Done Sent false
11585 3775 Acked Flush 0 Done Done Sent false
11586 3776 Acked Write 5 Done Done Sent false
11587 3777 Acked Write 5 Done Done Sent false
11588 3778 Acked Flush 0 Done Done Sent false
11589 3779 Acked Read 2 Done Done Sent false
11590 3780 Acked Flush 0 Done Done Sent false
11591 3781 Acked Read 7 Done Done Sent false
11592 3782 Acked Read 10 Done Done Sent false
11593 3783 Acked Read 5 Done Done Sent false
11594 3784 Acked Read 9 Done Done Sent false
11595 3785 Acked Write 8 Done Done Sent false
11596 3786 Acked Read 10 Done Done Sent false
11597 3787 Acked Write 7 Done Done Sent false
11598 3788 Acked Read 5 Done Done Sent false
11599 3789 Acked Read 1 Done Done Sent false
11600 3790 Acked Read 6 Done Done Sent false
11601 3791 Acked Write 2 Done Done Sent false
11602 3792 Acked Write 8 Done Done Sent false
11603 3793 Acked Read 9 Done Done Sent false
11604 3794 Acked Read 7 Done Done Sent false
11605 3795 Acked Read 10 Done Done Sent false
11606 3796 Acked Flush 0 Done Done Sent false
11607 3797 Acked Write 1 Done Done Sent false
11608 3798 Acked Write 5 Done Done Sent false
11609 3799 Acked Read 1 Done Done Sent false
11610 3800 Acked Read 3 Done Done Sent false
11611 3801 Acked Read 10 Done Done Sent false
11612 3802 Acked Flush 0 Done Done Sent false
11613 3803 Acked Read 3 Done Done Sent false
11614 3804 Acked Flush 0 Done Done Sent false
11615 3805 Acked Read 10 Done Done Sent false
11616 3806 Acked Write 1 Done Done Sent false
11617 3807 Acked Write 4 Done Done Sent false
11618 3808 Acked Read 6 Done Done Sent false
11619 3809 Acked Write 8 Done Done Sent false
11620 3810 Acked Read 10 Done Done Sent false
11621 3811 Acked Write 3 Done Done Sent false
11622 3812 Acked Read 3 Done Done Sent false
11623 3813 Acked Flush 0 Done Done Sent false
11624 3814 Acked Read 10 Done Done Sent false
11625 3815 Acked Read 7 Done Done Sent false
11626 3816 Acked Read 5 Done Done Sent false
11627 3817 Acked Read 9 Done Done Sent false
11628 3818 Acked Read 1 Done Done Sent false
11629 3819 Acked Read 4 Done Done Sent false
11630 3820 Acked Read 10 Done Done Sent false
11631 3821 Acked Flush 0 Done Done Sent false
11632 3822 Acked Read 4 Done Done Sent false
11633 3823 Acked Flush 0 Done Done Sent false
11634 3824 Acked Read 8 Done Done Sent false
11635 3825 Acked Write 2 Done Done Sent false
11636 3826 Acked Write 4 Done Done Sent false
11637 3827 Acked Read 1 Done Done Sent false
11638 3828 Acked Read 2 Done Done Sent false
11639 3829 Acked Read 9 Done Done Sent false
11640 3830 Acked Flush 0 Done Done Sent false
11641 3831 Acked Write 3 Done Done Sent false
11642 3832 Acked Read 1 Done Done Sent false
11643 3833 Acked Flush 0 Done Done Sent false
11644 3834 Acked Read 2 Done Done Sent false
11645 3835 Acked Write 10 Done Done Sent false
11646 3836 Acked Read 1 Done Done Sent false
11647 3837 Acked Flush 0 Done Done Sent false
11648 3838 Acked Read 1 Done Done Sent false
11649 3839 Acked Read 8 Done Done Sent false
11650 3840 Acked Write 8 Done Done Sent false
11651 3841 Acked Read 8 Done Done Sent false
11652 3842 Acked Write 10 Done Done Sent false
11653 3843 Acked Write 4 Done Done Sent false
11654 3844 Acked Read 2 Done Done Sent false
11655 3845 Acked Write 7 Done Done Sent false
11656 3846 Acked Write 10 Done Done Sent false
11657 3847 Acked Flush 0 Done Done Sent false
11658 3848 Acked Read 7 Done Done Sent false
11659 3849 Acked Read 7 Done Done Sent false
11660 3850 Acked Read 3 Done Done Sent false
11661 3851 Acked Read 8 Done Done Sent false
11662 3852 Acked Flush 0 Done Done Sent false
11663 3853 Acked Read 2 Done Done Sent false
11664 3854 Acked Read 5 Done Done Sent false
11665 3855 Acked Flush 0 Done Done Sent false
11666 3856 Acked Read 9 Done Done Sent false
11667 3857 Acked Write 4 Done Done Sent false
11668 3858 Acked Write 4 Done Done Sent false
11669 3859 Acked Write 1 Done Done Sent false
11670 3860 Acked Write 6 Done Done Sent false
11671 3861 Acked Read 8 Done Done Sent false
11672 3862 Acked Read 9 Done Done Sent false
11673 3863 Acked Read 3 Done Done Sent false
11674 3864 Acked Flush 0 Done Done Sent false
11675 3865 Acked Read 10 Done Done Sent false
11676 3866 Acked Read 10 Done Done Sent false
11677 3867 Acked Read 2 Done Done Sent false
11678 3868 Acked Write 2 Done Done Sent false
11679 3869 Acked Read 6 Done Done Sent false
11680 3870 Acked Read 2 Done Done Sent false
11681 3871 Acked Write 7 Done Done Sent false
11682 3872 Acked Write 6 Done Done Sent false
11683 3873 Acked Read 3 Done Done Sent false
11684 3874 Acked Read 5 Done Done Sent false
11685 3875 Acked Read 6 Done Done Sent false
11686 3876 Acked Write 5 Done Done Sent false
11687 3877 Acked Flush 0 Done Done Sent false
11688 3878 Acked Write 6 Done Done Sent false
11689 3879 Acked Read 9 Done Done Sent false
11690 3880 Acked Read 8 Done Done Sent false
11691 3881 Acked Read 8 Done Done Sent false
11692 3882 Acked Read 9 Done Done Sent false
11693 3883 Acked Read 2 Done Done Sent false
11694 3884 Acked Write 5 Done Done Sent false
11695 3885 Acked Write 7 Done Done Sent false
11696 3886 Acked Write 3 Done Done Sent false
11697 3887 Acked Write 9 Done Done Sent false
11698 3888 Acked Flush 0 Done Done Sent false
11699 3889 Acked Read 6 Done Done Sent false
11700 3890 Acked Read 3 Done Done Sent false
11701 3891 Acked Flush 0 Done Done Sent false
11702 3892 Acked Write 2 Done Done Sent false
11703 3893 Acked Read 5 Done Done Sent false
11704 3894 Acked Read 8 Done Done Sent false
11705 3895 Acked Write 10 Done Done Sent false
11706 3896 Acked Write 8 Done Done Sent false
11707 3897 Acked Read 5 Done Done Sent false
11708 3898 Acked Read 4 Done Done Sent false
11709 3899 Acked Read 7 Done Done Sent false
11710 3900 Acked Flush 0 Done Done Sent false
11711 3901 Acked Write 1 Done Done Sent false
11712 3902 Acked Read 2 Done Done Sent false
11713 3903 Acked Read 1 Done Done Sent false
11714 3904 Acked Read 8 Done Done Sent false
11715 3905 Acked Read 4 Done Done Sent false
11716 3906 Acked Write 2 Done Done Sent false
11717 3907 Acked Read 3 Done Done Sent false
11718 3908 Acked Flush 0 Done Done Sent false
11719 3909 Acked Read 1 Done Done Sent false
11720 3910 Acked Write 10 Done Done Sent false
11721 3911 Acked Read 1 Done Done Sent false
11722 3912 Acked Read 9 Done Done Sent false
11723 3913 Acked Read 9 Done Done Sent false
11724 3914 Acked Write 3 Done Done Sent false
11725 3915 Acked Write 10 Done Done Sent false
11726 3916 Acked Read 6 Done Done Sent false
11727 3917 Acked Flush 0 Done Done Sent false
11728 3918 Acked Read 8 Done Done Sent false
11729 3919 Acked Write 2 Done Done Sent false
11730 3920 Acked Flush 0 Done Done Sent false
11731 3921 Acked Read 6 Done Done Sent false
11732 3922 Acked Read 7 Done Done Sent false
11733 3923 Acked Read 5 Done Done Sent false
11734 3924 Acked Read 6 Done Done Sent false
11735 3925 Acked Write 9 Done Done Sent false
11736 3926 Acked Read 7 Done Done Sent false
11737 3927 Acked Write 9 Done Done Sent false
11738 3928 Acked Read 6 Done Done Sent false
11739 3929 Acked Read 9 Done Done Sent false
11740 3930 Acked Write 5 Done Done Sent false
11741 3931 Acked Read 2 Done Done Sent false
11742 3932 Acked Write 10 Done Done Sent false
11743 3933 Acked Read 9 Done Done Sent false
11744 3934 Acked Write 2 Done Done Sent false
11745 3935 Acked Read 7 Done Done Sent false
11746 3936 Acked Read 10 Done Done Sent false
11747 3937 Acked Write 1 Done Done Sent false
11748 3938 Acked Read 6 Done Done Sent false
11749 3939 Acked Read 2 Done Done Sent false
11750 3940 Acked Write 7 Done Done Sent false
11751 3941 Acked Read 4 Done Done Sent false
11752 3942 Acked Read 5 Done Done Sent false
11753 3943 Acked Read 4 Done Done Sent false
11754 3944 Acked Write 10 Done Done Sent false
11755 3945 Acked Read 7 Done Done Sent false
11756 3946 Acked Read 6 Done Done Sent false
11757 3947 Acked Write 1 Done Done Sent false
11758 3948 Acked Read 5 Done Done Sent false
11759 3949 Acked Read 9 Done Done Sent false
11760 3950 Acked Write 6 Done Done Sent false
11761 3951 Acked Write 6 Done Done Sent false
11762 3952 Acked Read 9 Done Done Sent false
11763 3953 Acked Read 5 Done Done Sent false
11764 3954 Acked Read 5 Done Done Sent false
11765 3955 Acked Read 1 Done Done Sent false
11766 3956 Acked Read 10 Done Done Sent false
11767 3957 Acked Read 3 Done Done Sent false
11768 3958 Acked Read 9 Done Done Sent false
11769 3959 Acked Flush 0 Done Done Sent false
11770 3960 Acked Flush 0 Done Done Sent false
11771 3961 Acked Read 7 Done Done Sent false
11772 3962 Acked Read 3 Done Done Sent false
11773 3963 Acked Flush 0 Done Done Sent false
11774 3964 Acked Write 8 Done Done Sent false
11775 3965 Acked Write 10 Done Done Sent false
11776 3966 Acked Read 6 Done Done Sent false
11777 3967 Acked Flush 0 Done Done Sent false
11778 3968 Acked Read 2 Done Done Sent false
11779 3969 Acked Flush 0 Done Done Sent false
11780 3970 Acked Read 3 Done Done Sent false
11781 3971 Acked Read 4 Done Done Sent false
11782 3972 Acked Read 4 Done Done Sent false
11783 3973 Acked Read 5 Done Done Sent false
11784 3974 Acked Write 7 Done Done Sent false
11785 3975 Acked Write 3 Done Done Sent false
11786 3976 Acked Write 6 Done Done Sent false
11787 3977 Acked Read 2 Done Done Sent false
11788 3978 Acked Write 1 Done Done Sent false
11789 3979 Acked Write 5 Done Done Sent false
11790 3980 Acked Read 3 Done Done Sent false
11791 3981 Acked Read 2 Done Done Sent false
11792 3982 Acked Read 8 Done Done Sent false
11793 3983 Acked Write 7 Done Done Sent false
11794 3984 Acked Write 3 Done Done Sent false
11795 3985 Acked Write 9 Done Done Sent false
11796 3986 Acked Flush 0 Done Done Sent false
11797 3987 Acked Read 10 Done Done Sent false
11798 3988 Acked Write 3 Done Done Sent false
11799 3989 Acked Write 6 Done Done Sent false
11800 3990 Acked Write 6 Done Done Sent false
11801 3991 Acked Write 9 Done Done Sent false
11802 3992 Acked Read 4 Done Done Sent false
11803 3993 Acked Flush 0 Done Done Sent false
11804 3994 Acked Flush 0 Done Done Sent false
11805 3995 Acked Flush 0 Done Done Sent false
11806 3996 Acked Read 2 Done Done Sent false
11807 3997 Acked Read 1 Done Done Sent false
11808 3998 Acked Read 7 Done Done Sent false
11809 3999 Acked Read 8 Done Done Sent false
11810 4000 Acked Read 5 Done Done Sent false
11811 4001 Acked Write 6 Done Done Sent false
11812 4002 Acked Write 4 Done Done Sent false
11813 4003 Acked Write 1 Done Done Sent false
11814 4004 Acked Write 6 Done Done Sent false
11815 4005 Acked Write 7 Done Done Sent false
11816 4006 Acked Read 6 Done Done Sent false
11817 4007 Acked Read 6 Done Done Sent false
11818 4008 Acked Write 9 Done Done Sent false
11819 4009 Acked Write 2 Done Done Sent false
11820 4010 Acked Write 6 Done Done Sent false
11821 4011 Acked Write 2 Done Done Sent false
11822 4012 Acked Write 6 Done Done Sent false
11823 4013 Acked Read 8 Done Done Sent false
11824 4014 Acked Read 8 Done Done Sent false
11825 4015 Acked Read 10 Done Done Sent false
11826 4016 Acked Read 8 Done Done Sent false
11827 4017 Acked Write 3 Done Done Sent false
11828 4018 Acked Write 6 Done Done Sent false
11829 4019 Acked Write 7 Done Done Sent false
11830 4020 Acked Write 7 Done Done Sent false
11831 4021 Acked Write 5 Done Done Sent false
11832 4022 Acked Read 1 Done Done Sent false
11833 4023 Acked Write 2 Done Done Sent false
11834 4024 Acked Flush 0 Done Done Sent false
11835 STATES DS:0 DS:1 DS:2 TOTAL
11836 Sent 0 0 303 303
11837 Done 303 303 0 606
11838 Skipped 0 0 0 0
11839 Error 0 0 0 0
11840 Last Flush: 4024 4024 3721
11841 Downstairs last five completed: 3721 3720 3719 3718 3717
11842 Upstairs last five completed: 4024 4023 4022 4021 4020
11843 CLIENT: Up:0 ds:303 act:2
118442024-11-07T14:05:46.469ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
118452024-11-07T14:05:46.469ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
118462024-11-07T14:05:46.469ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
118472024-11-07T14:05:46.469ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
118482024-11-07T14:05:46.469ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
11849 [8830] got start action
11850 [8830] I am going to restart
11851 Starting downstairs at port 8830
11852 Make output file at "/tmp/dsc/downstairs-8830.txt"
11853 [8830][2] reports Starting
11854 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1209)
11855 [8830][2] reports Running
11856 ----------------------------------------------------------------
11857 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
11858 States: Active Active Offline
11859 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11860 3722 Acked Read 4 Done Done Sent false
11861 3723 Acked Read 4 Done Done Sent false
11862 3724 Acked Read 7 Done Done Sent false
11863 3725 Acked Write 3 Done Done Sent false
11864 3726 Acked Write 1 Done Done Sent false
11865 3727 Acked Write 9 Done Done Sent false
11866 3728 Acked Write 8 Done Done Sent false
11867 3729 Acked Write 7 Done Done Sent false
11868 3730 Acked Read 10 Done Done Sent false
11869 3731 Acked Read 5 Done Done Sent false
11870 3732 Acked Write 2 Done Done Sent false
11871 3733 Acked Read 1 Done Done Sent false
11872 3734 Acked Write 3 Done Done Sent false
11873 3735 Acked Write 8 Done Done Sent false
11874 3736 Acked Flush 0 Done Done Sent false
11875 3737 Acked Read 3 Done Done Sent false
11876 3738 Acked Read 5 Done Done Sent false
11877 3739 Acked Read 7 Done Done Sent false
11878 3740 Acked Read 5 Done Done Sent false
11879 3741 Acked Read 7 Done Done Sent false
11880 3742 Acked Read 9 Done Done Sent false
11881 3743 Acked Read 9 Done Done Sent false
11882 3744 Acked Read 3 Done Done Sent false
11883 3745 Acked Read 10 Done Done Sent false
11884 3746 Acked Flush 0 Done Done Sent false
11885 3747 Acked Read 6 Done Done Sent false
11886 3748 Acked Write 10 Done Done Sent false
11887 3749 Acked Flush 0 Done Done Sent false
11888 3750 Acked Read 3 Done Done Sent false
11889 3751 Acked Read 10 Done Done Sent false
11890 3752 Acked Read 4 Done Done Sent false
11891 3753 Acked Read 2 Done Done Sent false
11892 3754 Acked Write 3 Done Done Sent false
11893 3755 Acked Write 1 Done Done Sent false
11894 3756 Acked Read 8 Done Done Sent false
11895 3757 Acked Write 9 Done Done Sent false
11896 3758 Acked Write 10 Done Done Sent false
11897 3759 Acked Read 1 Done Done Sent false
11898 3760 Acked Read 4 Done Done Sent false
11899 3761 Acked Read 4 Done Done Sent false
11900 3762 Acked Read 7 Done Done Sent false
11901 3763 Acked Write 3 Done Done Sent false
11902 3764 Acked Read 1 Done Done Sent false
11903 3765 Acked Write 9 Done Done Sent false
11904 3766 Acked Write 1 Done Done Sent false
11905 3767 Acked Write 3 Done Done Sent false
11906 3768 Acked Write 4 Done Done Sent false
11907 3769 Acked Write 9 Done Done Sent false
11908 3770 Acked Read 3 Done Done Sent false
11909 3771 Acked Write 1 Done Done Sent false
11910 3772 Acked Read 1 Done Done Sent false
11911 3773 Acked Read 10 Done Done Sent false
11912 3774 Acked Read 9 Done Done Sent false
11913 3775 Acked Flush 0 Done Done Sent false
11914 3776 Acked Write 5 Done Done Sent false
11915 3777 Acked Write 5 Done Done Sent false
11916 3778 Acked Flush 0 Done Done Sent false
11917 3779 Acked Read 2 Done Done Sent false
11918 3780 Acked Flush 0 Done Done Sent false
11919 3781 Acked Read 7 Done Done Sent false
11920 3782 Acked Read 10 Done Done Sent false
11921 3783 Acked Read 5 Done Done Sent false
11922 3784 Acked Read 9 Done Done Sent false
11923 3785 Acked Write 8 Done Done Sent false
11924 3786 Acked Read 10 Done Done Sent false
11925 3787 Acked Write 7 Done Done Sent false
11926 3788 Acked Read 5 Done Done Sent false
11927 3789 Acked Read 1 Done Done Sent false
11928 3790 Acked Read 6 Done Done Sent false
11929 3791 Acked Write 2 Done Done Sent false
11930 3792 Acked Write 8 Done Done Sent false
11931 3793 Acked Read 9 Done Done Sent false
11932 3794 Acked Read 7 Done Done Sent false
11933 3795 Acked Read 10 Done Done Sent false
11934 3796 Acked Flush 0 Done Done Sent false
11935 3797 Acked Write 1 Done Done Sent false
11936 3798 Acked Write 5 Done Done Sent false
11937 3799 Acked Read 1 Done Done Sent false
11938 3800 Acked Read 3 Done Done Sent false
11939 3801 Acked Read 10 Done Done Sent false
11940 3802 Acked Flush 0 Done Done Sent false
11941 3803 Acked Read 3 Done Done Sent false
11942 3804 Acked Flush 0 Done Done Sent false
11943 3805 Acked Read 10 Done Done Sent false
11944 3806 Acked Write 1 Done Done Sent false
11945 3807 Acked Write 4 Done Done Sent false
11946 3808 Acked Read 6 Done Done Sent false
11947 3809 Acked Write 8 Done Done Sent false
11948 3810 Acked Read 10 Done Done Sent false
11949 3811 Acked Write 3 Done Done Sent false
11950 3812 Acked Read 3 Done Done Sent false
11951 3813 Acked Flush 0 Done Done Sent false
11952 3814 Acked Read 10 Done Done Sent false
11953 3815 Acked Read 7 Done Done Sent false
11954 3816 Acked Read 5 Done Done Sent false
11955 3817 Acked Read 9 Done Done Sent false
11956 3818 Acked Read 1 Done Done Sent false
11957 3819 Acked Read 4 Done Done Sent false
11958 3820 Acked Read 10 Done Done Sent false
11959 3821 Acked Flush 0 Done Done Sent false
11960 3822 Acked Read 4 Done Done Sent false
11961 3823 Acked Flush 0 Done Done Sent false
11962 3824 Acked Read 8 Done Done Sent false
11963 3825 Acked Write 2 Done Done Sent false
11964 3826 Acked Write 4 Done Done Sent false
11965 3827 Acked Read 1 Done Done Sent false
11966 3828 Acked Read 2 Done Done Sent false
11967 3829 Acked Read 9 Done Done Sent false
11968 3830 Acked Flush 0 Done Done Sent false
11969 3831 Acked Write 3 Done Done Sent false
11970 3832 Acked Read 1 Done Done Sent false
11971 3833 Acked Flush 0 Done Done Sent false
11972 3834 Acked Read 2 Done Done Sent false
11973 3835 Acked Write 10 Done Done Sent false
11974 3836 Acked Read 1 Done Done Sent false
11975 3837 Acked Flush 0 Done Done Sent false
11976 3838 Acked Read 1 Done Done Sent false
11977 3839 Acked Read 8 Done Done Sent false
11978 3840 Acked Write 8 Done Done Sent false
11979 3841 Acked Read 8 Done Done Sent false
11980 3842 Acked Write 10 Done Done Sent false
11981 3843 Acked Write 4 Done Done Sent false
11982 3844 Acked Read 2 Done Done Sent false
11983 3845 Acked Write 7 Done Done Sent false
11984 3846 Acked Write 10 Done Done Sent false
11985 3847 Acked Flush 0 Done Done Sent false
11986 3848 Acked Read 7 Done Done Sent false
11987 3849 Acked Read 7 Done Done Sent false
11988 3850 Acked Read 3 Done Done Sent false
11989 3851 Acked Read 8 Done Done Sent false
11990 3852 Acked Flush 0 Done Done Sent false
11991 3853 Acked Read 2 Done Done Sent false
11992 3854 Acked Read 5 Done Done Sent false
11993 3855 Acked Flush 0 Done Done Sent false
11994 3856 Acked Read 9 Done Done Sent false
11995 3857 Acked Write 4 Done Done Sent false
11996 3858 Acked Write 4 Done Done Sent false
11997 3859 Acked Write 1 Done Done Sent false
11998 3860 Acked Write 6 Done Done Sent false
11999 3861 Acked Read 8 Done Done Sent false
12000 3862 Acked Read 9 Done Done Sent false
12001 3863 Acked Read 3 Done Done Sent false
12002 3864 Acked Flush 0 Done Done Sent false
12003 3865 Acked Read 10 Done Done Sent false
12004 3866 Acked Read 10 Done Done Sent false
12005 3867 Acked Read 2 Done Done Sent false
12006 3868 Acked Write 2 Done Done Sent false
12007 3869 Acked Read 6 Done Done Sent false
12008 3870 Acked Read 2 Done Done Sent false
12009 3871 Acked Write 7 Done Done Sent false
12010 3872 Acked Write 6 Done Done Sent false
12011 3873 Acked Read 3 Done Done Sent false
12012 3874 Acked Read 5 Done Done Sent false
12013 3875 Acked Read 6 Done Done Sent false
12014 3876 Acked Write 5 Done Done Sent false
12015 3877 Acked Flush 0 Done Done Sent false
12016 3878 Acked Write 6 Done Done Sent false
12017 3879 Acked Read 9 Done Done Sent false
12018 3880 Acked Read 8 Done Done Sent false
12019 3881 Acked Read 8 Done Done Sent false
12020 3882 Acked Read 9 Done Done Sent false
12021 3883 Acked Read 2 Done Done Sent false
12022 3884 Acked Write 5 Done Done Sent false
12023 3885 Acked Write 7 Done Done Sent false
12024 3886 Acked Write 3 Done Done Sent false
12025 3887 Acked Write 9 Done Done Sent false
12026 3888 Acked Flush 0 Done Done Sent false
12027 3889 Acked Read 6 Done Done Sent false
12028 3890 Acked Read 3 Done Done Sent false
12029 3891 Acked Flush 0 Done Done Sent false
12030 3892 Acked Write 2 Done Done Sent false
12031 3893 Acked Read 5 Done Done Sent false
12032 3894 Acked Read 8 Done Done Sent false
12033 3895 Acked Write 10 Done Done Sent false
12034 3896 Acked Write 8 Done Done Sent false
12035 3897 Acked Read 5 Done Done Sent false
12036 3898 Acked Read 4 Done Done Sent false
12037 3899 Acked Read 7 Done Done Sent false
12038 3900 Acked Flush 0 Done Done Sent false
12039 3901 Acked Write 1 Done Done Sent false
12040 3902 Acked Read 2 Done Done Sent false
12041 3903 Acked Read 1 Done Done Sent false
12042 3904 Acked Read 8 Done Done Sent false
12043 3905 Acked Read 4 Done Done Sent false
12044 3906 Acked Write 2 Done Done Sent false
12045 3907 Acked Read 3 Done Done Sent false
12046 3908 Acked Flush 0 Done Done Sent false
12047 3909 Acked Read 1 Done Done Sent false
12048 3910 Acked Write 10 Done Done Sent false
12049 3911 Acked Read 1 Done Done Sent false
12050 3912 Acked Read 9 Done Done Sent false
12051 3913 Acked Read 9 Done Done Sent false
12052 3914 Acked Write 3 Done Done Sent false
12053 3915 Acked Write 10 Done Done Sent false
12054 3916 Acked Read 6 Done Done Sent false
12055 3917 Acked Flush 0 Done Done Sent false
12056 3918 Acked Read 8 Done Done Sent false
12057 3919 Acked Write 2 Done Done Sent false
12058 3920 Acked Flush 0 Done Done Sent false
12059 3921 Acked Read 6 Done Done Sent false
12060 3922 Acked Read 7 Done Done Sent false
12061 3923 Acked Read 5 Done Done Sent false
12062 3924 Acked Read 6 Done Done Sent false
12063 3925 Acked Write 9 Done Done Sent false
12064 3926 Acked Read 7 Done Done Sent false
12065 3927 Acked Write 9 Done Done Sent false
12066 3928 Acked Read 6 Done Done Sent false
12067 3929 Acked Read 9 Done Done Sent false
12068 3930 Acked Write 5 Done Done Sent false
12069 3931 Acked Read 2 Done Done Sent false
12070 3932 Acked Write 10 Done Done Sent false
12071 3933 Acked Read 9 Done Done Sent false
12072 3934 Acked Write 2 Done Done Sent false
12073 3935 Acked Read 7 Done Done Sent false
12074 3936 Acked Read 10 Done Done Sent false
12075 3937 Acked Write 1 Done Done Sent false
12076 3938 Acked Read 6 Done Done Sent false
12077 3939 Acked Read 2 Done Done Sent false
12078 3940 Acked Write 7 Done Done Sent false
12079 3941 Acked Read 4 Done Done Sent false
12080 3942 Acked Read 5 Done Done Sent false
12081 3943 Acked Read 4 Done Done Sent false
12082 3944 Acked Write 10 Done Done Sent false
12083 3945 Acked Read 7 Done Done Sent false
12084 3946 Acked Read 6 Done Done Sent false
12085 3947 Acked Write 1 Done Done Sent false
12086 3948 Acked Read 5 Done Done Sent false
12087 3949 Acked Read 9 Done Done Sent false
12088 3950 Acked Write 6 Done Done Sent false
12089 3951 Acked Write 6 Done Done Sent false
12090 3952 Acked Read 9 Done Done Sent false
12091 3953 Acked Read 5 Done Done Sent false
12092 3954 Acked Read 5 Done Done Sent false
12093 3955 Acked Read 1 Done Done Sent false
12094 3956 Acked Read 10 Done Done Sent false
12095 3957 Acked Read 3 Done Done Sent false
12096 3958 Acked Read 9 Done Done Sent false
12097 3959 Acked Flush 0 Done Done Sent false
12098 3960 Acked Flush 0 Done Done Sent false
12099 3961 Acked Read 7 Done Done Sent false
12100 3962 Acked Read 3 Done Done Sent false
12101 3963 Acked Flush 0 Done Done Sent false
12102 3964 Acked Write 8 Done Done Sent false
12103 3965 Acked Write 10 Done Done Sent false
12104 3966 Acked Read 6 Done Done Sent false
12105 3967 Acked Flush 0 Done Done Sent false
12106 3968 Acked Read 2 Done Done Sent false
12107 3969 Acked Flush 0 Done Done Sent false
12108 3970 Acked Read 3 Done Done Sent false
12109 3971 Acked Read 4 Done Done Sent false
12110 3972 Acked Read 4 Done Done Sent false
12111 3973 Acked Read 5 Done Done Sent false
12112 3974 Acked Write 7 Done Done Sent false
12113 3975 Acked Write 3 Done Done Sent false
12114 3976 Acked Write 6 Done Done Sent false
12115 3977 Acked Read 2 Done Done Sent false
12116 3978 Acked Write 1 Done Done Sent false
12117 3979 Acked Write 5 Done Done Sent false
12118 3980 Acked Read 3 Done Done Sent false
12119 3981 Acked Read 2 Done Done Sent false
12120 3982 Acked Read 8 Done Done Sent false
12121 3983 Acked Write 7 Done Done Sent false
12122 3984 Acked Write 3 Done Done Sent false
12123 3985 Acked Write 9 Done Done Sent false
12124 3986 Acked Flush 0 Done Done Sent false
12125 3987 Acked Read 10 Done Done Sent false
12126 3988 Acked Write 3 Done Done Sent false
12127 3989 Acked Write 6 Done Done Sent false
12128 3990 Acked Write 6 Done Done Sent false
12129 3991 Acked Write 9 Done Done Sent false
12130 3992 Acked Read 4 Done Done Sent false
12131 3993 Acked Flush 0 Done Done Sent false
12132 3994 Acked Flush 0 Done Done Sent false
12133 3995 Acked Flush 0 Done Done Sent false
12134 3996 Acked Read 2 Done Done Sent false
12135 3997 Acked Read 1 Done Done Sent false
12136 3998 Acked Read 7 Done Done Sent false
12137 3999 Acked Read 8 Done Done Sent false
12138 4000 Acked Read 5 Done Done Sent false
12139 4001 Acked Write 6 Done Done Sent false
12140 4002 Acked Write 4 Done Done Sent false
12141 4003 Acked Write 1 Done Done Sent false
12142 4004 Acked Write 6 Done Done Sent false
12143 4005 Acked Write 7 Done Done Sent false
12144 4006 Acked Read 6 Done Done Sent false
12145 4007 Acked Read 6 Done Done Sent false
12146 4008 Acked Write 9 Done Done Sent false
12147 4009 Acked Write 2 Done Done Sent false
12148 4010 Acked Write 6 Done Done Sent false
12149 4011 Acked Write 2 Done Done Sent false
12150 4012 Acked Write 6 Done Done Sent false
12151 4013 Acked Read 8 Done Done Sent false
12152 4014 Acked Read 8 Done Done Sent false
12153 4015 Acked Read 10 Done Done Sent false
12154 4016 Acked Read 8 Done Done Sent false
12155 4017 Acked Write 3 Done Done Sent false
12156 4018 Acked Write 6 Done Done Sent false
12157 4019 Acked Write 7 Done Done Sent false
12158 4020 Acked Write 7 Done Done Sent false
12159 4021 Acked Write 5 Done Done Sent false
12160 4022 Acked Read 1 Done Done Sent false
12161 4023 Acked Write 2 Done Done Sent false
12162 4024 Acked Flush 0 Done Done Sent false
12163 STATES DS:0 DS:1 DS:2 TOTAL
12164 Sent 0 0 303 303
12165 Done 303 303 0 606
12166 Skipped 0 0 0 0
12167 Error 0 0 0 0
12168 Last Flush: 4024 4024 3721
12169 Downstairs last five completed: 3721 3720 3719 3718 3717
12170 Upstairs last five completed: 4024 4023 4022 4021 4020
12171 CLIENT: Up:0 ds:303 act:2
12172 ----------------------------------------------------------------
12173 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
12174 States: Active Active Offline
12175 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12176 3722 Acked Read 4 Done Done Sent false
12177 3723 Acked Read 4 Done Done Sent false
12178 3724 Acked Read 7 Done Done Sent false
12179 3725 Acked Write 3 Done Done Sent false
12180 3726 Acked Write 1 Done Done Sent false
12181 3727 Acked Write 9 Done Done Sent false
12182 3728 Acked Write 8 Done Done Sent false
12183 3729 Acked Write 7 Done Done Sent false
12184 3730 Acked Read 10 Done Done Sent false
12185 3731 Acked Read 5 Done Done Sent false
12186 3732 Acked Write 2 Done Done Sent false
12187 3733 Acked Read 1 Done Done Sent false
12188 3734 Acked Write 3 Done Done Sent false
12189 3735 Acked Write 8 Done Done Sent false
12190 3736 Acked Flush 0 Done Done Sent false
12191 3737 Acked Read 3 Done Done Sent false
12192 3738 Acked Read 5 Done Done Sent false
12193 3739 Acked Read 7 Done Done Sent false
12194 3740 Acked Read 5 Done Done Sent false
12195 3741 Acked Read 7 Done Done Sent false
12196 3742 Acked Read 9 Done Done Sent false
12197 3743 Acked Read 9 Done Done Sent false
12198 3744 Acked Read 3 Done Done Sent false
12199 3745 Acked Read 10 Done Done Sent false
12200 3746 Acked Flush 0 Done Done Sent false
12201 3747 Acked Read 6 Done Done Sent false
12202 3748 Acked Write 10 Done Done Sent false
12203 3749 Acked Flush 0 Done Done Sent false
12204 3750 Acked Read 3 Done Done Sent false
12205 3751 Acked Read 10 Done Done Sent false
12206 3752 Acked Read 4 Done Done Sent false
12207 3753 Acked Read 2 Done Done Sent false
12208 3754 Acked Write 3 Done Done Sent false
12209 3755 Acked Write 1 Done Done Sent false
12210 3756 Acked Read 8 Done Done Sent false
12211 3757 Acked Write 9 Done Done Sent false
12212 3758 Acked Write 10 Done Done Sent false
12213 3759 Acked Read 1 Done Done Sent false
12214 3760 Acked Read 4 Done Done Sent false
12215 3761 Acked Read 4 Done Done Sent false
12216 3762 Acked Read 7 Done Done Sent false
12217 3763 Acked Write 3 Done Done Sent false
12218 3764 Acked Read 1 Done Done Sent false
12219 3765 Acked Write 9 Done Done Sent false
12220 3766 Acked Write 1 Done Done Sent false
12221 3767 Acked Write 3 Done Done Sent false
12222 3768 Acked Write 4 Done Done Sent false
12223 3769 Acked Write 9 Done Done Sent false
12224 3770 Acked Read 3 Done Done Sent false
12225 3771 Acked Write 1 Done Done Sent false
12226 3772 Acked Read 1 Done Done Sent false
12227 3773 Acked Read 10 Done Done Sent false
12228 3774 Acked Read 9 Done Done Sent false
12229 3775 Acked Flush 0 Done Done Sent false
12230 3776 Acked Write 5 Done Done Sent false
12231 3777 Acked Write 5 Done Done Sent false
12232 3778 Acked Flush 0 Done Done Sent false
12233 3779 Acked Read 2 Done Done Sent false
12234 3780 Acked Flush 0 Done Done Sent false
12235 3781 Acked Read 7 Done Done Sent false
12236 3782 Acked Read 10 Done Done Sent false
12237 3783 Acked Read 5 Done Done Sent false
12238 3784 Acked Read 9 Done Done Sent false
12239 3785 Acked Write 8 Done Done Sent false
12240 3786 Acked Read 10 Done Done Sent false
12241 3787 Acked Write 7 Done Done Sent false
12242 3788 Acked Read 5 Done Done Sent false
12243 3789 Acked Read 1 Done Done Sent false
12244 3790 Acked Read 6 Done Done Sent false
12245 3791 Acked Write 2 Done Done Sent false
12246 3792 Acked Write 8 Done Done Sent false
12247 3793 Acked Read 9 Done Done Sent false
12248 3794 Acked Read 7 Done Done Sent false
12249 3795 Acked Read 10 Done Done Sent false
12250 3796 Acked Flush 0 Done Done Sent false
12251 3797 Acked Write 1 Done Done Sent false
12252 3798 Acked Write 5 Done Done Sent false
12253 3799 Acked Read 1 Done Done Sent false
12254 3800 Acked Read 3 Done Done Sent false
12255 3801 Acked Read 10 Done Done Sent false
12256 3802 Acked Flush 0 Done Done Sent false
12257 3803 Acked Read 3 Done Done Sent false
12258 3804 Acked Flush 0 Done Done Sent false
12259 3805 Acked Read 10 Done Done Sent false
12260 3806 Acked Write 1 Done Done Sent false
12261 3807 Acked Write 4 Done Done Sent false
12262 3808 Acked Read 6 Done Done Sent false
12263 3809 Acked Write 8 Done Done Sent false
12264 3810 Acked Read 10 Done Done Sent false
12265 3811 Acked Write 3 Done Done Sent false
12266 3812 Acked Read 3 Done Done Sent false
12267 3813 Acked Flush 0 Done Done Sent false
12268 3814 Acked Read 10 Done Done Sent false
12269 3815 Acked Read 7 Done Done Sent false
12270 3816 Acked Read 5 Done Done Sent false
12271 3817 Acked Read 9 Done Done Sent false
12272 3818 Acked Read 1 Done Done Sent false
12273 3819 Acked Read 4 Done Done Sent false
12274 3820 Acked Read 10 Done Done Sent false
12275 3821 Acked Flush 0 Done Done Sent false
12276 3822 Acked Read 4 Done Done Sent false
12277 3823 Acked Flush 0 Done Done Sent false
12278 3824 Acked Read 8 Done Done Sent false
12279 3825 Acked Write 2 Done Done Sent false
12280 3826 Acked Write 4 Done Done Sent false
12281 3827 Acked Read 1 Done Done Sent false
12282 3828 Acked Read 2 Done Done Sent false
12283 3829 Acked Read 9 Done Done Sent false
12284 3830 Acked Flush 0 Done Done Sent false
12285 3831 Acked Write 3 Done Done Sent false
12286 3832 Acked Read 1 Done Done Sent false
12287 3833 Acked Flush 0 Done Done Sent false
12288 3834 Acked Read 2 Done Done Sent false
12289 3835 Acked Write 10 Done Done Sent false
12290 3836 Acked Read 1 Done Done Sent false
12291 3837 Acked Flush 0 Done Done Sent false
12292 3838 Acked Read 1 Done Done Sent false
12293 3839 Acked Read 8 Done Done Sent false
12294 3840 Acked Write 8 Done Done Sent false
12295 3841 Acked Read 8 Done Done Sent false
12296 3842 Acked Write 10 Done Done Sent false
12297 3843 Acked Write 4 Done Done Sent false
12298 3844 Acked Read 2 Done Done Sent false
12299 3845 Acked Write 7 Done Done Sent false
12300 3846 Acked Write 10 Done Done Sent false
12301 3847 Acked Flush 0 Done Done Sent false
12302 3848 Acked Read 7 Done Done Sent false
12303 3849 Acked Read 7 Done Done Sent false
12304 3850 Acked Read 3 Done Done Sent false
12305 3851 Acked Read 8 Done Done Sent false
12306 3852 Acked Flush 0 Done Done Sent false
12307 3853 Acked Read 2 Done Done Sent false
12308 3854 Acked Read 5 Done Done Sent false
12309 3855 Acked Flush 0 Done Done Sent false
12310 3856 Acked Read 9 Done Done Sent false
12311 3857 Acked Write 4 Done Done Sent false
12312 3858 Acked Write 4 Done Done Sent false
12313 3859 Acked Write 1 Done Done Sent false
12314 3860 Acked Write 6 Done Done Sent false
12315 3861 Acked Read 8 Done Done Sent false
12316 3862 Acked Read 9 Done Done Sent false
12317 3863 Acked Read 3 Done Done Sent false
12318 3864 Acked Flush 0 Done Done Sent false
12319 3865 Acked Read 10 Done Done Sent false
12320 3866 Acked Read 10 Done Done Sent false
12321 3867 Acked Read 2 Done Done Sent false
12322 3868 Acked Write 2 Done Done Sent false
12323 3869 Acked Read 6 Done Done Sent false
12324 3870 Acked Read 2 Done Done Sent false
12325 3871 Acked Write 7 Done Done Sent false
12326 3872 Acked Write 6 Done Done Sent false
12327 3873 Acked Read 3 Done Done Sent false
12328 3874 Acked Read 5 Done Done Sent false
12329 3875 Acked Read 6 Done Done Sent false
12330 3876 Acked Write 5 Done Done Sent false
12331 3877 Acked Flush 0 Done Done Sent false
12332 3878 Acked Write 6 Done Done Sent false
12333 3879 Acked Read 9 Done Done Sent false
12334 3880 Acked Read 8 Done Done Sent false
12335 3881 Acked Read 8 Done Done Sent false
12336 3882 Acked Read 9 Done Done Sent false
12337 3883 Acked Read 2 Done Done Sent false
12338 3884 Acked Write 5 Done Done Sent false
12339 3885 Acked Write 7 Done Done Sent false
12340 3886 Acked Write 3 Done Done Sent false
12341 3887 Acked Write 9 Done Done Sent false
12342 3888 Acked Flush 0 Done Done Sent false
12343 3889 Acked Read 6 Done Done Sent false
12344 3890 Acked Read 3 Done Done Sent false
12345 3891 Acked Flush 0 Done Done Sent false
12346 3892 Acked Write 2 Done Done Sent false
12347 3893 Acked Read 5 Done Done Sent false
12348 3894 Acked Read 8 Done Done Sent false
12349 3895 Acked Write 10 Done Done Sent false
12350 3896 Acked Write 8 Done Done Sent false
12351 3897 Acked Read 5 Done Done Sent false
12352 3898 Acked Read 4 Done Done Sent false
12353 3899 Acked Read 7 Done Done Sent false
12354 3900 Acked Flush 0 Done Done Sent false
12355 3901 Acked Write 1 Done Done Sent false
12356 3902 Acked Read 2 Done Done Sent false
12357 3903 Acked Read 1 Done Done Sent false
12358 3904 Acked Read 8 Done Done Sent false
12359 3905 Acked Read 4 Done Done Sent false
12360 3906 Acked Write 2 Done Done Sent false
12361 3907 Acked Read 3 Done Done Sent false
12362 3908 Acked Flush 0 Done Done Sent false
12363 3909 Acked Read 1 Done Done Sent false
12364 3910 Acked Write 10 Done Done Sent false
12365 3911 Acked Read 1 Done Done Sent false
12366 3912 Acked Read 9 Done Done Sent false
12367 3913 Acked Read 9 Done Done Sent false
12368 3914 Acked Write 3 Done Done Sent false
12369 3915 Acked Write 10 Done Done Sent false
12370 3916 Acked Read 6 Done Done Sent false
12371 3917 Acked Flush 0 Done Done Sent false
12372 3918 Acked Read 8 Done Done Sent false
12373 3919 Acked Write 2 Done Done Sent false
12374 3920 Acked Flush 0 Done Done Sent false
12375 3921 Acked Read 6 Done Done Sent false
12376 3922 Acked Read 7 Done Done Sent false
12377 3923 Acked Read 5 Done Done Sent false
12378 3924 Acked Read 6 Done Done Sent false
12379 3925 Acked Write 9 Done Done Sent false
12380 3926 Acked Read 7 Done Done Sent false
12381 3927 Acked Write 9 Done Done Sent false
12382 3928 Acked Read 6 Done Done Sent false
12383 3929 Acked Read 9 Done Done Sent false
12384 3930 Acked Write 5 Done Done Sent false
12385 3931 Acked Read 2 Done Done Sent false
12386 3932 Acked Write 10 Done Done Sent false
12387 3933 Acked Read 9 Done Done Sent false
12388 3934 Acked Write 2 Done Done Sent false
12389 3935 Acked Read 7 Done Done Sent false
12390 3936 Acked Read 10 Done Done Sent false
12391 3937 Acked Write 1 Done Done Sent false
12392 3938 Acked Read 6 Done Done Sent false
12393 3939 Acked Read 2 Done Done Sent false
12394 3940 Acked Write 7 Done Done Sent false
12395 3941 Acked Read 4 Done Done Sent false
12396 3942 Acked Read 5 Done Done Sent false
12397 3943 Acked Read 4 Done Done Sent false
12398 3944 Acked Write 10 Done Done Sent false
12399 3945 Acked Read 7 Done Done Sent false
12400 3946 Acked Read 6 Done Done Sent false
12401 3947 Acked Write 1 Done Done Sent false
12402 3948 Acked Read 5 Done Done Sent false
12403 3949 Acked Read 9 Done Done Sent false
12404 3950 Acked Write 6 Done Done Sent false
12405 3951 Acked Write 6 Done Done Sent false
12406 3952 Acked Read 9 Done Done Sent false
12407 3953 Acked Read 5 Done Done Sent false
12408 3954 Acked Read 5 Done Done Sent false
12409 3955 Acked Read 1 Done Done Sent false
12410 3956 Acked Read 10 Done Done Sent false
12411 3957 Acked Read 3 Done Done Sent false
12412 3958 Acked Read 9 Done Done Sent false
12413 3959 Acked Flush 0 Done Done Sent false
12414 3960 Acked Flush 0 Done Done Sent false
12415 3961 Acked Read 7 Done Done Sent false
12416 3962 Acked Read 3 Done Done Sent false
12417 3963 Acked Flush 0 Done Done Sent false
12418 3964 Acked Write 8 Done Done Sent false
12419 3965 Acked Write 10 Done Done Sent false
12420 3966 Acked Read 6 Done Done Sent false
12421 3967 Acked Flush 0 Done Done Sent false
12422 3968 Acked Read 2 Done Done Sent false
12423 3969 Acked Flush 0 Done Done Sent false
12424 3970 Acked Read 3 Done Done Sent false
12425 3971 Acked Read 4 Done Done Sent false
12426 3972 Acked Read 4 Done Done Sent false
12427 3973 Acked Read 5 Done Done Sent false
12428 3974 Acked Write 7 Done Done Sent false
12429 3975 Acked Write 3 Done Done Sent false
12430 3976 Acked Write 6 Done Done Sent false
12431 3977 Acked Read 2 Done Done Sent false
12432 3978 Acked Write 1 Done Done Sent false
12433 3979 Acked Write 5 Done Done Sent false
12434 3980 Acked Read 3 Done Done Sent false
12435 3981 Acked Read 2 Done Done Sent false
12436 3982 Acked Read 8 Done Done Sent false
12437 3983 Acked Write 7 Done Done Sent false
12438 3984 Acked Write 3 Done Done Sent false
12439 3985 Acked Write 9 Done Done Sent false
12440 3986 Acked Flush 0 Done Done Sent false
12441 3987 Acked Read 10 Done Done Sent false
12442 3988 Acked Write 3 Done Done Sent false
12443 3989 Acked Write 6 Done Done Sent false
12444 3990 Acked Write 6 Done Done Sent false
12445 3991 Acked Write 9 Done Done Sent false
12446 3992 Acked Read 4 Done Done Sent false
12447 3993 Acked Flush 0 Done Done Sent false
12448 3994 Acked Flush 0 Done Done Sent false
12449 3995 Acked Flush 0 Done Done Sent false
12450 3996 Acked Read 2 Done Done Sent false
12451 3997 Acked Read 1 Done Done Sent false
12452 3998 Acked Read 7 Done Done Sent false
12453 3999 Acked Read 8 Done Done Sent false
12454 4000 Acked Read 5 Done Done Sent false
12455 4001 Acked Write 6 Done Done Sent false
12456 4002 Acked Write 4 Done Done Sent false
12457 4003 Acked Write 1 Done Done Sent false
12458 4004 Acked Write 6 Done Done Sent false
12459 4005 Acked Write 7 Done Done Sent false
12460 4006 Acked Read 6 Done Done Sent false
12461 4007 Acked Read 6 Done Done Sent false
12462 4008 Acked Write 9 Done Done Sent false
12463 4009 Acked Write 2 Done Done Sent false
12464 4010 Acked Write 6 Done Done Sent false
12465 4011 Acked Write 2 Done Done Sent false
12466 4012 Acked Write 6 Done Done Sent false
12467 4013 Acked Read 8 Done Done Sent false
12468 4014 Acked Read 8 Done Done Sent false
12469 4015 Acked Read 10 Done Done Sent false
12470 4016 Acked Read 8 Done Done Sent false
12471 4017 Acked Write 3 Done Done Sent false
12472 4018 Acked Write 6 Done Done Sent false
12473 4019 Acked Write 7 Done Done Sent false
12474 4020 Acked Write 7 Done Done Sent false
12475 4021 Acked Write 5 Done Done Sent false
12476 4022 Acked Read 1 Done Done Sent false
12477 4023 Acked Write 2 Done Done Sent false
12478 4024 Acked Flush 0 Done Done Sent false
12479 STATES DS:0 DS:1 DS:2 TOTAL
12480 Sent 0 0 303 303
12481 Done 303 303 0 606
12482 Skipped 0 0 0 0
12483 Error 0 0 0 0
12484 Last Flush: 4024 4024 3721
12485 Downstairs last five completed: 3721 3720 3719 3718 3717
12486 Upstairs last five completed: 4024 4023 4022 4021 4020
12487 CLIENT: Up:0 ds:303 act:2
124882024-11-07T14:05:56.470ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
124892024-11-07T14:05:56.470ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38240) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
124902024-11-07T14:05:56.471ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
124912024-11-07T14:05:56.472ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
124922024-11-07T14:05:56.472ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
124932024-11-07T14:05:56.472ZINFOcrucible: send last flush ID to this DS: 3721 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
124942024-11-07T14:05:56.472ZINFOcrucible: Replied this last flush ID: 3721 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
124952024-11-07T14:05:56.472ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
124962024-11-07T14:05:56.472ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
124972024-11-07T14:05:56.472ZINFOcrucible: [2] client re-new 303 jobs since flush 3721 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
124982024-11-07T14:05:56.472ZINFOcrucible: [2] Replayed 303 jobs since flush: 3721 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
12499 ----------------------------------------------------------------
12500 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12501 Downstairs last five completed: 4024 4023 4022 4021 4020
12502 Upstairs last five completed: 4024 4023 4022 4021 4020
12503 CLIENT: Up:0 ds:0 act:3
12504 Replay: All jobs finished
12505 Main task has work to do, go find it
12506 got dsc Stop(0)
12507 stop 0
12508 Nov 07 14:05:56.762 INFO request completed, latency_us: 353, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 6b610cb6-9766-4cd6-a9d1-f903f219f218, remote_addr: [8810] Got stop action so:false kr:false
12509 127.0.0.1:34100, local_addr: 127.0.0.1:9998
125102024-11-07T14:05:56.762ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
125112024-11-07T14:05:56.762ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
125122024-11-07T14:05:56.762ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
125132024-11-07T14:05:56.762ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
125142024-11-07T14:05:56.762ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
125152024-11-07T14:05:56.763ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
12516 Nov 07 14:05:56.763 INFO request completed, latency_us: 138, response_code: 200, uri: /state/cid/0, method: GET, req_id: 99c5fe5e-1323-47a6-92ce-7f21889dbc61, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
12517 [8810] Exited with: ExitStatus(unix_wait_status(9))
12518 [8810][0] reports Exit
12519 Nov 07 14:06:00.765 INFO request completed, latency_us: 211, response_code: 200, uri: /state/cid/0, method: GET, req_id: b9de74d2-6c94-449e-aafb-56aa1255411b, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
12520 001/300 Write block 2162 len 20480 data: 3 4 4 4 4
12521 002/300 Read block 2333 len 40960
12522 003/300 Read block 3849 len 40960
12523 004/300 Write block 1029 len 4096 data: 3
12524 005/300 Read block 3245 len 40960
12525 006/300 Read block 3547 len 24576
12526 007/300 Read block 421 len 32768
12527 008/300 Write block 1656 len 24576 data: 4 4 2 3 4 4
12528 009/300 Write block 1083 len 4096 data: 6
12529 010/300 Read block 1204 len 16384
12530 011/300 Write block 999 len 12288 data: 2 3 3
12531 012/300 Write block 4021 len 16384 data: 4 4 4 3
12532 013/300 Read block 1374 len 28672
12533 014/300 Read block 2257 len 36864
12534 015/300 Read block 4343 len 24576
12535 016/300 Flush
12536 017/300 Read block 4128 len 16384
12537 018/300 Write block 4301 len 4096 data: 4
12538 019/300 Read block 1072 len 4096
12539 020/300 Write block 3399 len 32768 data: 3 4 5 5 5 4 2 2
12540 021/300 Read block 2765 len 12288
12541 022/300 Flush
12542 023/300 Read block 1953 len 12288
12543 024/300 Write block 538 len 36864 data: 3 3 3 2 2 4 3 3 3
12544 025/300 Read block 4698 len 32768
12545 026/300 Read block 1554 len 28672
12546 027/300 Read block 106 len 28672
12547 028/300 Flush
12548 029/300 Write block 4787 len 16384 data: 3 4 4 4
12549 030/300 Read block 2610 len 28672
12550 031/300 Read block 3519 len 20480
12551 032/300 Read block 3059 len 12288
12552 033/300 Read block 2581 len 40960
12553 034/300 Write block 4034 len 12288 data: 2 2 2
12554 035/300 Write block 2667 len 24576 data: 2 3 3 3 3 3
12555 036/300 Read block 1164 len 8192
12556 037/300 Write block 1715 len 28672 data: 3 3 3 3 3 4 3
12557 038/300 Read block 2167 len 16384
12558 039/300 Write block 362 len 8192 data: 2 2
12559 040/300 Write block 1986 len 16384 data: 4 4 4 3
12560 041/300 Read block 1574 len 4096
12561 042/300 Read block 3094 len 4096
12562 043/300 Read block 4274 len 32768
12563 044/300 Write block 3240 len 40960 data: 4 4 4 3 3 3 3 3 4 3
12564 045/300 Write block 3618 len 32768 data: 6 6 6 6 6 6 5 4
12565 046/300 Write block 927 len 16384 data: 2 3 2 2
12566 047/300 Read block 2417 len 8192
12567 048/300 Read block 511 len 16384
12568 049/300 Write block 3292 len 32768 data: 5 4 4 4 4 4 4 2
12569 050/300 Write block 4967 len 16384 data: 4 4 3 3
12570 051/300 Write block 4277 len 24576 data: 3 3 3 4 4 2
12571 052/300 Flush
12572 053/300 Read block 208 len 20480
12573 054/300 Read block 3070 len 12288
12574 055/300 Write block 3744 len 4096 data: 3
12575 056/300 Write block 2624 len 8192 data: 3 2
12576 057/300 Read block 3678 len 32768
12577 058/300 Read block 4543 len 16384
12578 059/300 Read block 4877 len 20480
12579 060/300 Read block 1651 len 4096
12580 061/300 Read block 574 len 20480
12581 062/300 Read block 1594 len 40960
12582 063/300 Read block 730 len 36864
12583 064/300 Write block 2709 len 12288 data: 2 2 2
12584 065/300 Read block 2018 len 20480
12585 066/300 Write block 1944 len 28672 data: 4 4 4 3 3 3 3
12586 067/300 Flush
12587 068/300 Write block 756 len 4096 data: 4
12588 069/300 Write block 1250 len 16384 data: 3 3 3 3
12589 070/300 Read block 1992 len 32768
12590 071/300 Flush
12591 072/300 Write block 3835 len 8192 data: 4 4
12592 073/300 Read block 2704 len 12288
12593 074/300 Read block 4852 len 36864
12594 075/300 Write block 68 len 4096 data: 2
12595 076/300 Read block 3903 len 20480
12596 077/300 Read block 1639 len 36864
12597 078/300 Write block 2609 len 28672 data: 3 3 3 3 3 4 3
12598 079/300 Write block 514 len 40960 data: 3 4 4 3 4 4 4 4 3 2
12599 080/300 Read block 4797 len 16384
12600 081/300 Read block 2925 len 24576
12601 082/300 Read block 4215 len 16384
12602 083/300 Write block 768 len 32768 data: 2 2 2 3 3 3 3 3
12603 084/300 Write block 3567 len 32768 data: 4 4 4 3 2 2 2 2
12604 085/300 Write block 2350 len 24576 data: 2 2 2 2 2 2
12605 086/300 Write block 2873 len 36864 data: 2 3 2 2 2 2 3 4 4
12606 087/300 Read block 328 len 4096
12607 088/300 Read block 3036 len 24576
12608 089/300 Write block 1309 len 36864 data: 3 3 3 3 3 3 2 2 3
12609 090/300 Write block 2720 len 8192 data: 4 5
12610 091/300 Write block 4288 len 32768 data: 4 3 3 3 4 4 4 4
12611 092/300 Write block 4433 len 28672 data: 4 4 5 5 4 5 6
12612 093/300 Read block 1618 len 40960
12613 094/300 Write block 3554 len 12288 data: 4 4 3
12614 095/300 Write block 2696 len 32768 data: 4 4 4 4 4 2 2 2
12615 096/300 Read block 2257 len 16384
12616 097/300 Write block 737 len 20480 data: 2 2 2 3 3
12617 098/300 Write block 4882 len 16384 data: 2 2 2 2
12618 099/300 Read block 3933 len 28672
12619 100/300 Write block 4526 len 36864 data: 5 5 5 3 3 3 4 3 4
12620 101/300 Write block 3168 len 4096 data: 4
12621 102/300 Read block 263 len 24576
12622 103/300 Write block 2617 len 8192 data: 3 3
12623 104/300 Flush
12624 105/300 Write block 719 len 16384 data: 2 2 3 4
12625 106/300 Read block 3950 len 12288
12626 107/300 Write block 1569 len 32768 data: 3 3 2 3 4 4 4 4
12627 108/300 Write block 3552 len 40960 data: 5 5 5 5 4 2 4 4 3 4
12628 109/300 Read block 4665 len 40960
12629 110/300 Write block 4174 len 40960 data: 5 4 4 4 4 3 2 2 2 2
12630 111/300 Read block 1467 len 36864
12631 112/300 Read block 673 len 20480
12632 113/300 Write block 3047 len 32768 data: 2 3 5 5 5 5 5 4
12633 114/300 Read block 2212 len 12288
12634 115/300 Read block 2233 len 12288
12635 116/300 Write block 686 len 4096 data: 2
12636 117/300 Flush
12637 118/300 Flush
12638 119/300 Write block 3598 len 24576 data: 4 4 4 4 4 4
12639 120/300 Read block 1603 len 20480
12640 121/300 Read block 4335 len 24576
12641 122/300 Write block 1335 len 16384 data: 4 4 4 3
12642 123/300 Write block 4894 len 16384 data: 3 4 5 4
12643 124/300 Read block 584 len 24576
12644 125/300 Read block 3077 len 32768
12645 126/300 Write block 3151 len 4096 data: 3
12646 127/300 Flush
12647 128/300 Read block 1654 len 32768
12648 129/300 Read block 3801 len 24576
12649 130/300 Read block 249 len 36864
12650 131/300 Read block 991 len 20480
12651 132/300 Write block 2793 len 24576 data: 3 3 2 2 2 2
12652 133/300 Read block 792 len 16384
12653 134/300 Read block 403 len 12288
12654 135/300 Read block 2156 len 40960
12655 136/300 Read block 4156 len 24576
12656 137/300 Read block 1691 len 4096
12657 138/300 Read block 3085 len 12288
12658 139/300 Read block 74 len 12288
12659 140/300 Write block 1234 len 32768 data: 4 4 3 3 3 3 3 3
12660 141/300 Read block 622 len 16384
12661 142/300 Read block 3639 len 24576
12662 143/300 Read block 1757 len 28672
12663 144/300 Read block 4598 len 28672
12664 145/300 Write block 677 len 16384 data: 2 3 3 3
12665 146/300 Write block 2 len 4096 data: 3
12666 147/300 Read block 735 len 28672
12667 148/300 Read block 2108 len 36864
12668 149/300 Flush
12669 150/300 Write block 728 len 36864 data: 5 4 4 3 4 4 3 3 2
12670 151/300 Read block 3573 len 32768
12671 152/300 Write block 4212 len 8192 data: 3 3
12672 153/300 Flush
12673 154/300 Read block 721 len 32768
12674 155/300 Read block 837 len 20480
12675 156/300 Write block 4891 len 24576 data: 2 3 3 4 5 6
12676 157/300 Write block 728 len 8192 data: 6 5
12677 158/300 Flush
12678 159/300 Read block 2489 len 16384
12679 160/300 Read block 2671 len 36864
12680 161/300 Read block 15 len 28672
12681 162/300 Write block 3234 len 28672 data: 4 4 5 4 4 3 5
12682 163/300 Write block 4957 len 20480 data: 5 4 4 4 3
12683 164/300 Write block 3904 len 12288 data: 5 4 4
12684 165/300 Read block 3991 len 32768
12685 166/300 Read block 2055 len 20480
12686 167/300 Read block 3801 len 12288
12687 168/300 Write block 4687 len 20480 data: 4 3 3 3 2
12688 169/300 Read block 3655 len 32768
12689 170/300 Flush
12690 171/300 Read block 157 len 32768
12691 172/300 Read block 731 len 4096
12692 173/300 Read block 2254 len 32768
12693 174/300 Read block 632 len 12288
12694 175/300 Flush
12695 176/300 Write block 2665 len 8192 data: 4 3
12696 177/300 Read block 4010 len 4096
12697 178/300 Write block 4375 len 32768 data: 3 2 2 2 2 2 2 3
12698 179/300 Read block 137 len 8192
12699 180/300 Read block 1099 len 24576
12700 181/300 Write block 2251 len 40960 data: 3 3 2 2 2 2 2 2 2 2
12701 182/300 Read block 3836 len 8192
12702 183/300 Write block 4150 len 20480 data: 6 5 4 5 4
12703 184/300 Read block 838 len 12288
12704 185/300 Flush
12705 186/300 Write block 3477 len 40960 data: 5 5 5 5 4 4 5 5 3 4
12706 187/300 Read block 3283 len 4096
12707 188/300 Read block 2973 len 4096
12708 189/300 Read block 3759 len 32768
12709 190/300 Flush
12710 191/300 Write block 510 len 24576 data: 2 2 3 3 4 5
12711 192/300 Write block 1353 len 40960 data: 4 6 5 7 7 6 7 7 5 4
12712 193/300 Write block 962 len 16384 data: 2 2 2 2
12713 194/300 Read block 2613 len 36864
12714 195/300 Read block 319 len 28672
12715 196/300 Read block 206 len 4096
12716 197/300 Read block 413 len 36864
12717 198/300 Write block 3985 len 12288 data: 2 2 3
12718 199/300 Read block 752 len 40960
12719 200/300 Read block 1845 len 40960
12720 201/300 Write block 2241 len 12288 data: 5 3 2
12721 202/300 Write block 3693 len 28672 data: 5 5 5 4 4 5 4
12722 203/300 Read block 2986 len 4096
12723 204/300 Read block 1269 len 32768
12724 205/300 Read block 4684 len 12288
12725 206/300 Read block 4508 len 28672
12726 207/300 Write block 3826 len 8192 data: 3 3
12727 208/300 Read block 1458 len 32768
12728 209/300 Read block 2466 len 8192
12729 210/300 Write block 4442 len 20480 data: 4 4 4 4 3
12730 211/300 Read block 3813 len 20480
12731 212/300 Write block 1314 len 36864 data: 4 3 3 4 3 3 3 3 3
12732 213/300 Read block 2305 len 28672
12733 214/300 Read block 992 len 24576
12734 215/300 Read block 1736 len 16384
12735 216/300 Write block 1769 len 4096 data: 5
12736 217/300 Write block 2670 len 12288 data: 4 4 4
12737 218/300 Write block 224 len 16384 data: 2 2 3 3
12738 219/300 Read block 2207 len 32768
12739 220/300 Write block 1247 len 16384 data: 2 3 4 4
12740 221/300 Write block 3599 len 8192 data: 5 5
12741 222/300 Write block 2942 len 28672 data: 2 2 2 2 2 2 2
12742 223/300 Read block 48 len 4096
12743 224/300 Read block 1965 len 32768
12744 225/300 Write block 1883 len 36864 data: 4 5 4 4 3 3 3 3 3
12745 226/300 Write block 168 len 36864 data: 3 3 2 2 2 2 2 2 2
12746 227/300 Read block 3395 len 4096
12747 228/300 Write block 1741 len 24576 data: 2 3 3 3 3 3
12748 229/300 Write block 1023 len 24576 data: 2 2 2 2 2 2
12749 230/300 Read block 131 len 36864
12750 231/300 Write block 1960 len 32768 data: 3 3 2 2 2 2 3 4
12751 232/300 Write block 240 len 40960 data: 5 4 4 4 4 3 3 3 2 3
12752 233/300 Write block 3394 len 12288 data: 4 4 3
12753 234/300 Read block 3461 len 24576
12754 235/300 Read block 1733 len 40960
12755 236/300 Flush
12756 237/300 Read block 1273 len 24576
12757 238/300 Flush
12758 239/300 Read block 1027 len 12288
12759 240/300 Write block 4620 len 4096 data: 2
12760 241/300 Read block 4067 len 4096
12761 242/300 Flush
12762 243/300 Flush
12763 244/300 Write block 3932 len 16384 data: 3 3 3 3
12764 245/300 Read block 1182 len 28672
12765 246/300 Write block 3398 len 36864 data: 2 4 5 6 6 6 5 3 3
12766 247/300 Read block 659 len 8192
12767 248/300 Read block 3795 len 12288
12768 249/300 Write block 3644 len 4096 data: 2
12769 250/300 Flush
12770 251/300 Write block 3761 len 28672 data: 4 4 4 4 3 3 2
12771 252/300 Write block 1982 len 32768 data: 5 4 5 4 5 5 5 4
12772 253/300 Read block 1800 len 12288
12773 254/300 Write block 1750 len 28672 data: 5 5 4 4 3 3 3
12774 255/300 Read block 2876 len 32768
12775 256/300 Read block 2651 len 36864
12776 257/300 Read block 1731 len 16384
12777 258/300 Write block 3702 len 28672 data: 5 6 6 4 3 3 3
12778 259/300 Write block 4461 len 36864 data: 2 3 2 3 3 3 3 3 3
12779 260/300 Read block 3983 len 32768
12780 261/300 Write block 2130 len 24576 data: 3 3 4 4 4 3
12781 262/300 Write block 1791 len 28672 data: 5 5 4 5 5 3 3
12782 263/300 Read block 2342 len 32768
12783 264/300 Read block 2116 len 24576
12784 265/300 Flush
12785 266/300 Write block 1210 len 40960 data: 3 3 3 3 3 2 2 2 2 2
12786 267/300 Read block 3474 len 32768
12787 268/300 Read block 2652 len 36864
12788 269/300 Flush
12789 270/300 Write block 2316 len 16384 data: 2 2 3 2
12790 271/300 Read block 277 len 16384
12791 272/300 Read block 3664 len 32768
12792 273/300 Flush
12793 274/300 Write block 4159 len 4096 data: 2
12794 275/300 Flush
12795 276/300 Read block 4450 len 4096
12796 277/300 Write block 1429 len 40960 data: 3 3 3 3 3 3 2 3 4 4
12797 278/300 Read block 2681 len 24576
12798 279/300 Read block 4352 len 12288
12799 280/300 Read block 1337 len 36864
12800 281/300 Read block 230 len 12288
12801 282/300 Read block 4654 len 40960
12802 283/300 Read block 54 len 40960
12803 284/300 Read block 1562 len 24576
12804 285/300 Read block 1708 len 4096
12805 286/300 Read block 772 len 4096
12806 287/300 Read block 3943 len 36864
12807 288/300 Write block 1762 len 20480 data: 2 2 2 2 2
12808 289/300 Flush
12809 290/300 Read block 3920 len 36864
12810 291/300 Read block 3425 len 36864
12811 292/300 Flush
12812 293/300 Write block 1278 len 24576 data: 2 2 2 2 2 2
12813 294/300 Write block 4543 len 20480 data: 3 3 3 3 3
12814 295/300 Write block 732 len 24576 data: 5 5 4 4 3 3
12815 296/300 Flush
12816 297/300 Write block 3344 len 16384 data: 4 5 4 3
12817 298/300 Write block 4260 len 28672 data: 3 3 3 3 3 3 3
12818 299/300 Read block 1069 len 24576
12819 300/300 Flush
12820 301/300 Write block 292 len 16384 data: 3 4 4 4
12821 Main task has work to do, go find it
12822 got dsc Start(0)
12823 start 0
12824 Nov 07 14:06:01.062 INFO request completed, latency_us: 208, response_code: 204, uri: /start/cid/0, method: POST, req_id: 1f96abb8-77e0-4c3e-bb44-e11af07b03f7, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
12825 Replay: started 0, returned:Ok(())
12826 ----------------------------------------------------------------
12827 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12828 States: Offline Active Active
12829 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12830 4025 Acked Write 5 Sent Done Done false
12831 4026 Acked Read 10 Sent Done Done false
12832 4027 Acked Read 10 Sent Done Done false
12833 4028 Acked Write 1 Sent Done Done false
12834 4029 Acked Read 10 Sent Done Done false
12835 4030 Acked Read 6 Sent Done Done false
12836 4031 Acked Read 8 Sent Done Done false
12837 4032 Acked Write 6 Sent Done Done false
12838 4033 Acked Write 1 Sent Done Done false
12839 4034 Acked Read 4 Sent Done Done false
12840 4035 Acked Write 3 Sent Done Done false
12841 4036 Acked Write 4 Sent Done Done false
12842 4037 Acked Read 7 Sent Done Done false
12843 4038 Acked Read 9 Sent Done Done false
12844 4039 Acked Read 6 Sent Done Done false
12845 4040 Acked Flush 0 Sent Done Done false
12846 4041 Acked Read 4 Sent Done Done false
12847 4042 Acked Write 1 Sent Done Done false
12848 4043 Acked Read 1 Sent Done Done false
12849 4044 Acked Write 8 Sent Done Done false
12850 4045 Acked Read 3 Sent Done Done false
12851 4046 Acked Flush 0 Sent Done Done false
12852 4047 Acked Read 3 Sent Done Done false
12853 4048 Acked Write 9 Sent Done Done false
12854 4049 Acked Read 8 Sent Done Done false
12855 4050 Acked Read 7 Sent Done Done false
12856 4051 Acked Read 7 Sent Done Done false
12857 4052 Acked Flush 0 Sent Done Done false
12858 4053 Acked Write 4 Sent Done Done false
12859 4054 Acked Read 7 Sent Done Done false
12860 4055 Acked Read 5 Sent Done Done false
12861 4056 Acked Read 3 Sent Done Done false
12862 4057 Acked Read 10 Sent Done Done false
12863 4058 Acked Write 3 Sent Done Done false
12864 4059 Acked Write 6 Sent Done Done false
12865 4060 Acked Read 2 Sent Done Done false
12866 4061 Acked Write 7 Sent Done Done false
12867 4062 Acked Read 4 Sent Done Done false
12868 4063 Acked Write 2 Sent Done Done false
12869 4064 Acked Write 4 Sent Done Done false
12870 4065 Acked Read 1 Sent Done Done false
12871 4066 Acked Read 1 Sent Done Done false
12872 4067 Acked Read 8 Sent Done Done false
12873 4068 Acked Write 10 Sent Done Done false
12874 4069 Acked Write 8 Sent Done Done false
12875 4070 Acked Write 4 Sent Done Done false
12876 4071 Acked Read 2 Sent Done Done false
12877 4072 Acked Read 4 Sent Done Done false
12878 4073 Acked Write 8 Sent Done Done false
12879 4074 Acked Write 4 Sent Done Done false
12880 4075 Acked Write 6 Sent Done Done false
12881 4076 Acked Flush 0 Sent Done Done false
12882 4077 Acked Read 5 Sent Done Done false
12883 4078 Acked Read 3 Sent Done Done false
12884 4079 Acked Write 1 Sent Done Done false
12885 4080 Acked Write 2 Sent Done Done false
12886 4081 Acked Read 8 Sent Done Done false
12887 4082 Acked Read 4 Sent Done Done false
12888 4083 Acked Read 5 Sent Done Done false
12889 4084 Acked Read 1 Sent Done Done false
12890 4085 Acked Read 5 Sent Done Done false
12891 4086 Acked Read 10 Sent Done Done false
12892 4087 Acked Read 9 Sent Done Done false
12893 4088 Acked Write 3 Sent Done Done false
12894 4089 Acked Read 5 Sent Done Done false
12895 4090 Acked Write 7 Sent Done Done false
12896 4091 Acked Flush 0 Sent Done Done false
12897 4092 Acked Write 1 Sent Done Done false
12898 4093 Acked Write 4 Sent Done Done false
12899 4094 Acked Read 8 Sent Done Done false
12900 4095 Acked Flush 0 Sent Done Done false
12901 4096 Acked Write 2 Sent Done Done false
12902 4097 Acked Read 3 Sent Done Done false
12903 4098 Acked Read 9 Sent Done Done false
12904 4099 Acked Write 1 Sent Done Done false
12905 4100 Acked Read 5 Sent Done Done false
12906 4101 Acked Read 9 Sent Done Done false
12907 4102 Acked Write 7 Sent Done Done false
12908 4103 Acked Write 10 Sent Done Done false
12909 4104 Acked Read 4 Sent Done Done false
12910 4105 Acked Read 6 Sent Done Done false
12911 4106 Acked Read 4 Sent Done Done false
12912 4107 Acked Write 8 Sent Done Done false
12913 4108 Acked Write 8 Sent Done Done false
12914 4109 Acked Write 6 Sent Done Done false
12915 4110 Acked Write 9 Sent Done Done false
12916 4111 Acked Read 1 Sent Done Done false
12917 4112 Acked Read 6 Sent Done Done false
12918 4113 Acked Write 9 Sent Done Done false
12919 4114 Acked Write 2 Sent Done Done false
12920 4115 Acked Write 8 Sent Done Done false
12921 4116 Acked Write 7 Sent Done Done false
12922 4117 Acked Read 10 Sent Done Done false
12923 4118 Acked Write 3 Sent Done Done false
12924 4119 Acked Write 8 Sent Done Done false
12925 4120 Acked Read 4 Sent Done Done false
12926 4121 Acked Write 5 Sent Done Done false
12927 4122 Acked Write 4 Sent Done Done false
12928 4123 Acked Read 7 Sent Done Done false
12929 4124 Acked Write 9 Sent Done Done false
12930 4125 Acked Write 1 Sent Done Done false
12931 4126 Acked Read 6 Sent Done Done false
12932 4127 Acked Write 2 Sent Done Done false
12933 4128 Acked Flush 0 Sent Done Done false
12934 4129 Acked Write 4 Sent Done Done false
12935 4130 Acked Read 3 Sent Done Done false
12936 4131 Acked Write 8 Sent Done Done false
12937 4132 Acked Write 10 Sent Done Done false
12938 4133 Acked Read 10 Sent Done Done false
12939 4134 Acked Write 10 Sent Done Done false
12940 4135 Acked Read 9 Sent Done Done false
12941 4136 Acked Read 5 Sent Done Done false
12942 4137 Acked Write 8 Sent Done Done false
12943 4138 Acked Read 3 Sent Done Done false
12944 4139 Acked Read 3 Sent Done Done false
12945 4140 Acked Write 1 Sent Done Done false
12946 4141 Acked Flush 0 Sent Done Done false
12947 4142 Acked Flush 0 Sent Done Done false
12948 4143 Acked Write 6 Sent Done Done false
12949 4144 Acked Read 5 Sent Done Done false
12950 4145 Acked Read 6 Sent Done Done false
12951 4146 Acked Write 4 Sent Done Done false
12952 4147 Acked Write 4 Sent Done Done false
12953 4148 Acked Read 6 Sent Done Done false
12954 4149 Acked Read 8 Sent Done Done false
12955 4150 Acked Write 1 Sent Done Done false
12956 4151 Acked Flush 0 Sent Done Done false
12957 4152 Acked Read 8 Sent Done Done false
12958 4153 Acked Read 6 Sent Done Done false
12959 4154 Acked Read 9 Sent Done Done false
12960 4155 Acked Read 5 Sent Done Done false
12961 4156 Acked Write 6 Sent Done Done false
12962 4157 Acked Read 4 Sent Done Done false
12963 4158 Acked Read 3 Sent Done Done false
12964 4159 Acked Read 10 Sent Done Done false
12965 4160 Acked Read 6 Sent Done Done false
12966 4161 Acked Read 1 Sent Done Done false
12967 4162 Acked Read 3 Sent Done Done false
12968 4163 Acked Read 3 Sent Done Done false
12969 4164 Acked Write 8 Sent Done Done false
12970 4165 Acked Read 4 Sent Done Done false
12971 4166 Acked Read 6 Sent Done Done false
12972 4167 Acked Read 7 Sent Done Done false
12973 4168 Acked Read 7 Sent Done Done false
12974 4169 Acked Write 4 Sent Done Done false
12975 4170 Acked Write 1 Sent Done Done false
12976 4171 Acked Read 7 Sent Done Done false
12977 4172 Acked Read 9 Sent Done Done false
12978 4173 Acked Flush 0 Sent Done Done false
12979 4174 Acked Write 9 Sent Done Done false
12980 4175 Acked Read 8 Sent Done Done false
12981 4176 Acked Write 2 Sent Done Done false
12982 4177 Acked Flush 0 Sent Done Done false
12983 4178 Acked Read 8 Sent Done Done false
12984 4179 Acked Read 5 Sent Done Done false
12985 4180 Acked Write 6 Sent Done Done false
12986 4181 Acked Write 2 Sent Done Done false
12987 4182 Acked Flush 0 Sent Done Done false
12988 4183 Acked Read 4 Sent Done Done false
12989 4184 Acked Read 9 Sent Done Done false
12990 4185 Acked Read 7 Sent Done Done false
12991 4186 Acked Write 7 Sent Done Done false
12992 4187 Acked Write 5 Sent Done Done false
12993 4188 Acked Write 3 Sent Done Done false
12994 4189 Acked Read 8 Sent Done Done false
12995 4190 Acked Read 5 Sent Done Done false
12996 4191 Acked Read 3 Sent Done Done false
12997 4192 Acked Write 5 Sent Done Done false
12998 4193 Acked Read 8 Sent Done Done false
12999 4194 Acked Flush 0 Sent Done Done false
13000 4195 Acked Read 8 Sent Done Done false
13001 4196 Acked Read 1 Sent Done Done false
13002 4197 Acked Read 8 Sent Done Done false
13003 4198 Acked Read 3 Sent Done Done false
13004 4199 Acked Flush 0 Sent Done Done false
13005 4200 Acked Write 2 Sent Done Done false
13006 4201 Acked Read 1 Sent Done Done false
13007 4202 Acked Write 8 Sent Done Done false
13008 4203 Acked Read 2 Sent Done Done false
13009 4204 Acked Read 6 Sent Done Done false
13010 4205 Acked Write 10 Sent Done Done false
13011 4206 Acked Read 2 Sent Done Done false
13012 4207 Acked Write 5 Sent Done Done false
13013 4208 Acked Read 3 Sent Done Done false
13014 4209 Acked Flush 0 Sent Done Done false
13015 4210 Acked Write 10 Sent Done Done false
13016 4211 Acked Read 1 Sent Done Done false
13017 4212 Acked Read 1 Sent Done Done false
13018 4213 Acked Read 8 Sent Done Done false
13019 4214 Acked Flush 0 Sent Done Done false
13020 4215 Acked Write 6 Sent Done Done false
13021 4216 Acked Write 10 Sent Done Done false
13022 4217 Acked Write 4 Sent Done Done false
13023 4218 Acked Read 9 Sent Done Done false
13024 4219 Acked Read 7 Sent Done Done false
13025 4220 Acked Read 1 Sent Done Done false
13026 4221 Acked Read 9 Sent Done Done false
13027 4222 Acked Write 3 Sent Done Done false
13028 4223 Acked Read 10 Sent Done Done false
13029 4224 Acked Read 10 Sent Done Done false
13030 4225 Acked Write 3 Sent Done Done false
13031 4226 Acked Write 7 Sent Done Done false
13032 4227 Acked Read 1 Sent Done Done false
13033 4228 Acked Read 8 Sent Done Done false
13034 4229 Acked Read 3 Sent Done Done false
13035 4230 Acked Read 7 Sent Done Done false
13036 4231 Acked Write 2 Sent Done Done false
13037 4232 Acked Read 8 Sent Done Done false
13038 4233 Acked Read 2 Sent Done Done false
13039 4234 Acked Write 5 Sent Done Done false
13040 4235 Acked Read 5 Sent Done Done false
13041 4236 Acked Write 9 Sent Done Done false
13042 4237 Acked Read 7 Sent Done Done false
13043 4238 Acked Read 6 Sent Done Done false
13044 4239 Acked Read 4 Sent Done Done false
13045 4240 Acked Write 1 Sent Done Done false
13046 4241 Acked Write 3 Sent Done Done false
13047 4242 Acked Write 4 Sent Done Done false
13048 4243 Acked Read 8 Sent Done Done false
13049 4244 Acked Write 4 Sent Done Done false
13050 4245 Acked Write 2 Sent Done Done false
13051 4246 Acked Write 7 Sent Done Done false
13052 4247 Acked Read 1 Sent Done Done false
13053 4248 Acked Read 8 Sent Done Done false
13054 4249 Acked Write 9 Sent Done Done false
13055 4250 Acked Write 9 Sent Done Done false
13056 4251 Acked Read 1 Sent Done Done false
13057 4252 Acked Write 6 Sent Done Done false
13058 4253 Acked Write 6 Sent Done Done false
13059 4254 Acked Read 9 Sent Done Done false
13060 4255 Acked Write 8 Sent Done Done false
13061 4256 Acked Write 10 Sent Done Done false
13062 4257 Acked Write 3 Sent Done Done false
13063 4258 Acked Read 6 Sent Done Done false
13064 4259 Acked Read 10 Sent Done Done false
13065 4260 Acked Flush 0 Sent Done Done false
13066 4261 Acked Read 6 Sent Done Done false
13067 4262 Acked Flush 0 Sent Done Done false
13068 4263 Acked Read 3 Sent Done Done false
13069 4264 Acked Write 1 Sent Done Done false
13070 4265 Acked Read 1 Sent Done Done false
13071 4266 Acked Flush 0 Sent Done Done false
13072 4267 Acked Flush 0 Sent Done Done false
13073 4268 Acked Write 4 Sent Done Done false
13074 4269 Acked Read 7 Sent Done Done false
13075 4270 Acked Write 9 Sent Done Done false
13076 4271 Acked Read 2 Sent Done Done false
13077 4272 Acked Read 3 Sent Done Done false
13078 4273 Acked Write 1 Sent Done Done false
13079 4274 Acked Flush 0 Sent Done Done false
13080 4275 Acked Write 7 Sent Done Done false
13081 4276 Acked Write 8 Sent Done Done false
13082 4277 Acked Read 3 Sent Done Done false
13083 4278 Acked Write 7 Sent Done Done false
13084 4279 Acked Read 8 Sent Done Done false
13085 4280 Acked Read 9 Sent Done Done false
13086 4281 Acked Read 4 Sent Done Done false
13087 4282 Acked Write 7 Sent Done Done false
13088 4283 Acked Write 9 Sent Done Done false
13089 4284 Acked Read 8 Sent Done Done false
13090 4285 Acked Write 6 Sent Done Done false
13091 4286 Acked Write 7 Sent Done Done false
13092 4287 Acked Read 8 Sent Done Done false
13093 4288 Acked Read 6 Sent Done Done false
13094 4289 Acked Flush 0 Sent Done Done false
13095 4290 Acked Write 10 Sent Done Done false
13096 4291 Acked Read 8 Sent Done Done false
13097 4292 Acked Read 9 Sent Done Done false
13098 4293 Acked Flush 0 Sent Done Done false
13099 4294 Acked Write 4 Sent Done Done false
13100 4295 Acked Read 4 Sent Done Done false
13101 4296 Acked Read 8 Sent Done Done false
13102 4297 Acked Flush 0 Sent Done Done false
13103 4298 Acked Write 1 Sent Done Done false
13104 4299 Acked Flush 0 Sent Done Done false
13105 4300 Acked Read 1 Sent Done Done false
13106 4301 Acked Write 10 Sent Done Done false
13107 4302 Acked Read 6 Sent Done Done false
13108 4303 Acked Read 3 Sent Done Done false
13109 4304 Acked Read 9 Sent Done Done false
13110 4305 Acked Read 3 Sent Done Done false
13111 4306 Acked Read 10 Sent Done Done false
13112 4307 Acked Read 10 Sent Done Done false
13113 4308 Acked Read 6 Sent Done Done false
13114 4309 Acked Read 1 Sent Done Done false
13115 4310 Acked Read 1 Sent Done Done false
13116 4311 Acked Read 9 Sent Done Done false
13117 4312 Acked Write 5 Sent Done Done false
13118 4313 Acked Flush 0 Sent Done Done false
13119 4314 Acked Read 9 Sent Done Done false
13120 4315 Acked Read 9 Sent Done Done false
13121 4316 Acked Flush 0 Sent Done Done false
13122 4317 Acked Write 6 Sent Done Done false
13123 4318 Acked Write 5 Sent Done Done false
13124 4319 Acked Write 6 Sent Done Done false
13125 4320 Acked Flush 0 Sent Done Done false
13126 4321 Acked Write 4 Sent Done Done false
13127 4322 Acked Write 7 Sent Done Done false
13128 4323 Acked Read 6 Sent Done Done false
13129 4324 Acked Flush 0 Sent Done Done false
13130 4325 Acked Write 4 Sent Done Done false
13131 STATES DS:0 DS:1 DS:2 TOTAL
13132 Sent 301 0 0 301
13133 Done 0 301 301 602
13134 Skipped 0 0 0 0
13135 Error 0 0 0 0
13136 Last Flush: 4024 4324 4324
13137 Downstairs last five completed: 4024 4023 4022 4021 4020
13138 Upstairs last five completed: 4325 4324 4323 4322 4321
13139 CLIENT: Up:0 ds:301 act:2
13140 [8810] got start action
13141 [8810] I am going to restart
13142 Starting downstairs at port 8810
13143 Make output file at "/tmp/dsc/downstairs-8810.txt"
13144 [8810][0] reports Starting
13145 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1210)
13146 [8810][0] reports Running
13147 ----------------------------------------------------------------
13148 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13149 States: Offline Active Active
13150 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13151 4025 Acked Write 5 Sent Done Done false
13152 4026 Acked Read 10 Sent Done Done false
13153 4027 Acked Read 10 Sent Done Done false
13154 4028 Acked Write 1 Sent Done Done false
13155 4029 Acked Read 10 Sent Done Done false
13156 4030 Acked Read 6 Sent Done Done false
13157 4031 Acked Read 8 Sent Done Done false
13158 4032 Acked Write 6 Sent Done Done false
13159 4033 Acked Write 1 Sent Done Done false
13160 4034 Acked Read 4 Sent Done Done false
13161 4035 Acked Write 3 Sent Done Done false
13162 4036 Acked Write 4 Sent Done Done false
13163 4037 Acked Read 7 Sent Done Done false
13164 4038 Acked Read 9 Sent Done Done false
13165 4039 Acked Read 6 Sent Done Done false
13166 4040 Acked Flush 0 Sent Done Done false
13167 4041 Acked Read 4 Sent Done Done false
13168 4042 Acked Write 1 Sent Done Done false
13169 4043 Acked Read 1 Sent Done Done false
13170 4044 Acked Write 8 Sent Done Done false
13171 4045 Acked Read 3 Sent Done Done false
13172 4046 Acked Flush 0 Sent Done Done false
13173 4047 Acked Read 3 Sent Done Done false
13174 4048 Acked Write 9 Sent Done Done false
13175 4049 Acked Read 8 Sent Done Done false
13176 4050 Acked Read 7 Sent Done Done false
13177 4051 Acked Read 7 Sent Done Done false
13178 4052 Acked Flush 0 Sent Done Done false
13179 4053 Acked Write 4 Sent Done Done false
13180 4054 Acked Read 7 Sent Done Done false
13181 4055 Acked Read 5 Sent Done Done false
13182 4056 Acked Read 3 Sent Done Done false
13183 4057 Acked Read 10 Sent Done Done false
13184 4058 Acked Write 3 Sent Done Done false
13185 4059 Acked Write 6 Sent Done Done false
13186 4060 Acked Read 2 Sent Done Done false
13187 4061 Acked Write 7 Sent Done Done false
13188 4062 Acked Read 4 Sent Done Done false
13189 4063 Acked Write 2 Sent Done Done false
13190 4064 Acked Write 4 Sent Done Done false
13191 4065 Acked Read 1 Sent Done Done false
13192 4066 Acked Read 1 Sent Done Done false
13193 4067 Acked Read 8 Sent Done Done false
13194 4068 Acked Write 10 Sent Done Done false
13195 4069 Acked Write 8 Sent Done Done false
13196 4070 Acked Write 4 Sent Done Done false
13197 4071 Acked Read 2 Sent Done Done false
13198 4072 Acked Read 4 Sent Done Done false
13199 4073 Acked Write 8 Sent Done Done false
13200 4074 Acked Write 4 Sent Done Done false
13201 4075 Acked Write 6 Sent Done Done false
13202 4076 Acked Flush 0 Sent Done Done false
13203 4077 Acked Read 5 Sent Done Done false
13204 4078 Acked Read 3 Sent Done Done false
13205 4079 Acked Write 1 Sent Done Done false
13206 4080 Acked Write 2 Sent Done Done false
13207 4081 Acked Read 8 Sent Done Done false
13208 4082 Acked Read 4 Sent Done Done false
13209 4083 Acked Read 5 Sent Done Done false
13210 4084 Acked Read 1 Sent Done Done false
13211 4085 Acked Read 5 Sent Done Done false
13212 4086 Acked Read 10 Sent Done Done false
13213 4087 Acked Read 9 Sent Done Done false
13214 4088 Acked Write 3 Sent Done Done false
13215 4089 Acked Read 5 Sent Done Done false
13216 4090 Acked Write 7 Sent Done Done false
13217 4091 Acked Flush 0 Sent Done Done false
13218 4092 Acked Write 1 Sent Done Done false
13219 4093 Acked Write 4 Sent Done Done false
13220 4094 Acked Read 8 Sent Done Done false
13221 4095 Acked Flush 0 Sent Done Done false
13222 4096 Acked Write 2 Sent Done Done false
13223 4097 Acked Read 3 Sent Done Done false
13224 4098 Acked Read 9 Sent Done Done false
13225 4099 Acked Write 1 Sent Done Done false
13226 4100 Acked Read 5 Sent Done Done false
13227 4101 Acked Read 9 Sent Done Done false
13228 4102 Acked Write 7 Sent Done Done false
13229 4103 Acked Write 10 Sent Done Done false
13230 4104 Acked Read 4 Sent Done Done false
13231 4105 Acked Read 6 Sent Done Done false
13232 4106 Acked Read 4 Sent Done Done false
13233 4107 Acked Write 8 Sent Done Done false
13234 4108 Acked Write 8 Sent Done Done false
13235 4109 Acked Write 6 Sent Done Done false
13236 4110 Acked Write 9 Sent Done Done false
13237 4111 Acked Read 1 Sent Done Done false
13238 4112 Acked Read 6 Sent Done Done false
13239 4113 Acked Write 9 Sent Done Done false
13240 4114 Acked Write 2 Sent Done Done false
13241 4115 Acked Write 8 Sent Done Done false
13242 4116 Acked Write 7 Sent Done Done false
13243 4117 Acked Read 10 Sent Done Done false
13244 4118 Acked Write 3 Sent Done Done false
13245 4119 Acked Write 8 Sent Done Done false
13246 4120 Acked Read 4 Sent Done Done false
13247 4121 Acked Write 5 Sent Done Done false
13248 4122 Acked Write 4 Sent Done Done false
13249 4123 Acked Read 7 Sent Done Done false
13250 4124 Acked Write 9 Sent Done Done false
13251 4125 Acked Write 1 Sent Done Done false
13252 4126 Acked Read 6 Sent Done Done false
13253 4127 Acked Write 2 Sent Done Done false
13254 4128 Acked Flush 0 Sent Done Done false
13255 4129 Acked Write 4 Sent Done Done false
13256 4130 Acked Read 3 Sent Done Done false
13257 4131 Acked Write 8 Sent Done Done false
13258 4132 Acked Write 10 Sent Done Done false
13259 4133 Acked Read 10 Sent Done Done false
13260 4134 Acked Write 10 Sent Done Done false
13261 4135 Acked Read 9 Sent Done Done false
13262 4136 Acked Read 5 Sent Done Done false
13263 4137 Acked Write 8 Sent Done Done false
13264 4138 Acked Read 3 Sent Done Done false
13265 4139 Acked Read 3 Sent Done Done false
13266 4140 Acked Write 1 Sent Done Done false
13267 4141 Acked Flush 0 Sent Done Done false
13268 4142 Acked Flush 0 Sent Done Done false
13269 4143 Acked Write 6 Sent Done Done false
13270 4144 Acked Read 5 Sent Done Done false
13271 4145 Acked Read 6 Sent Done Done false
13272 4146 Acked Write 4 Sent Done Done false
13273 4147 Acked Write 4 Sent Done Done false
13274 4148 Acked Read 6 Sent Done Done false
13275 4149 Acked Read 8 Sent Done Done false
13276 4150 Acked Write 1 Sent Done Done false
13277 4151 Acked Flush 0 Sent Done Done false
13278 4152 Acked Read 8 Sent Done Done false
13279 4153 Acked Read 6 Sent Done Done false
13280 4154 Acked Read 9 Sent Done Done false
13281 4155 Acked Read 5 Sent Done Done false
13282 4156 Acked Write 6 Sent Done Done false
13283 4157 Acked Read 4 Sent Done Done false
13284 4158 Acked Read 3 Sent Done Done false
13285 4159 Acked Read 10 Sent Done Done false
13286 4160 Acked Read 6 Sent Done Done false
13287 4161 Acked Read 1 Sent Done Done false
13288 4162 Acked Read 3 Sent Done Done false
13289 4163 Acked Read 3 Sent Done Done false
13290 4164 Acked Write 8 Sent Done Done false
13291 4165 Acked Read 4 Sent Done Done false
13292 4166 Acked Read 6 Sent Done Done false
13293 4167 Acked Read 7 Sent Done Done false
13294 4168 Acked Read 7 Sent Done Done false
13295 4169 Acked Write 4 Sent Done Done false
13296 4170 Acked Write 1 Sent Done Done false
13297 4171 Acked Read 7 Sent Done Done false
13298 4172 Acked Read 9 Sent Done Done false
13299 4173 Acked Flush 0 Sent Done Done false
13300 4174 Acked Write 9 Sent Done Done false
13301 4175 Acked Read 8 Sent Done Done false
13302 4176 Acked Write 2 Sent Done Done false
13303 4177 Acked Flush 0 Sent Done Done false
13304 4178 Acked Read 8 Sent Done Done false
13305 4179 Acked Read 5 Sent Done Done false
13306 4180 Acked Write 6 Sent Done Done false
13307 4181 Acked Write 2 Sent Done Done false
13308 4182 Acked Flush 0 Sent Done Done false
13309 4183 Acked Read 4 Sent Done Done false
13310 4184 Acked Read 9 Sent Done Done false
13311 4185 Acked Read 7 Sent Done Done false
13312 4186 Acked Write 7 Sent Done Done false
13313 4187 Acked Write 5 Sent Done Done false
13314 4188 Acked Write 3 Sent Done Done false
13315 4189 Acked Read 8 Sent Done Done false
13316 4190 Acked Read 5 Sent Done Done false
13317 4191 Acked Read 3 Sent Done Done false
13318 4192 Acked Write 5 Sent Done Done false
13319 4193 Acked Read 8 Sent Done Done false
13320 4194 Acked Flush 0 Sent Done Done false
13321 4195 Acked Read 8 Sent Done Done false
13322 4196 Acked Read 1 Sent Done Done false
13323 4197 Acked Read 8 Sent Done Done false
13324 4198 Acked Read 3 Sent Done Done false
13325 4199 Acked Flush 0 Sent Done Done false
13326 4200 Acked Write 2 Sent Done Done false
13327 4201 Acked Read 1 Sent Done Done false
13328 4202 Acked Write 8 Sent Done Done false
13329 4203 Acked Read 2 Sent Done Done false
13330 4204 Acked Read 6 Sent Done Done false
13331 4205 Acked Write 10 Sent Done Done false
13332 4206 Acked Read 2 Sent Done Done false
13333 4207 Acked Write 5 Sent Done Done false
13334 4208 Acked Read 3 Sent Done Done false
13335 4209 Acked Flush 0 Sent Done Done false
13336 4210 Acked Write 10 Sent Done Done false
13337 4211 Acked Read 1 Sent Done Done false
13338 4212 Acked Read 1 Sent Done Done false
13339 4213 Acked Read 8 Sent Done Done false
13340 4214 Acked Flush 0 Sent Done Done false
13341 4215 Acked Write 6 Sent Done Done false
13342 4216 Acked Write 10 Sent Done Done false
13343 4217 Acked Write 4 Sent Done Done false
13344 4218 Acked Read 9 Sent Done Done false
13345 4219 Acked Read 7 Sent Done Done false
13346 4220 Acked Read 1 Sent Done Done false
13347 4221 Acked Read 9 Sent Done Done false
13348 4222 Acked Write 3 Sent Done Done false
13349 4223 Acked Read 10 Sent Done Done false
13350 4224 Acked Read 10 Sent Done Done false
13351 4225 Acked Write 3 Sent Done Done false
13352 4226 Acked Write 7 Sent Done Done false
13353 4227 Acked Read 1 Sent Done Done false
13354 4228 Acked Read 8 Sent Done Done false
13355 4229 Acked Read 3 Sent Done Done false
13356 4230 Acked Read 7 Sent Done Done false
13357 4231 Acked Write 2 Sent Done Done false
13358 4232 Acked Read 8 Sent Done Done false
13359 4233 Acked Read 2 Sent Done Done false
13360 4234 Acked Write 5 Sent Done Done false
13361 4235 Acked Read 5 Sent Done Done false
13362 4236 Acked Write 9 Sent Done Done false
13363 4237 Acked Read 7 Sent Done Done false
13364 4238 Acked Read 6 Sent Done Done false
13365 4239 Acked Read 4 Sent Done Done false
13366 4240 Acked Write 1 Sent Done Done false
13367 4241 Acked Write 3 Sent Done Done false
13368 4242 Acked Write 4 Sent Done Done false
13369 4243 Acked Read 8 Sent Done Done false
13370 4244 Acked Write 4 Sent Done Done false
13371 4245 Acked Write 2 Sent Done Done false
13372 4246 Acked Write 7 Sent Done Done false
13373 4247 Acked Read 1 Sent Done Done false
13374 4248 Acked Read 8 Sent Done Done false
13375 4249 Acked Write 9 Sent Done Done false
13376 4250 Acked Write 9 Sent Done Done false
13377 4251 Acked Read 1 Sent Done Done false
13378 4252 Acked Write 6 Sent Done Done false
13379 4253 Acked Write 6 Sent Done Done false
13380 4254 Acked Read 9 Sent Done Done false
13381 4255 Acked Write 8 Sent Done Done false
13382 4256 Acked Write 10 Sent Done Done false
13383 4257 Acked Write 3 Sent Done Done false
13384 4258 Acked Read 6 Sent Done Done false
13385 4259 Acked Read 10 Sent Done Done false
13386 4260 Acked Flush 0 Sent Done Done false
13387 4261 Acked Read 6 Sent Done Done false
13388 4262 Acked Flush 0 Sent Done Done false
13389 4263 Acked Read 3 Sent Done Done false
13390 4264 Acked Write 1 Sent Done Done false
13391 4265 Acked Read 1 Sent Done Done false
13392 4266 Acked Flush 0 Sent Done Done false
13393 4267 Acked Flush 0 Sent Done Done false
13394 4268 Acked Write 4 Sent Done Done false
13395 4269 Acked Read 7 Sent Done Done false
13396 4270 Acked Write 9 Sent Done Done false
13397 4271 Acked Read 2 Sent Done Done false
13398 4272 Acked Read 3 Sent Done Done false
13399 4273 Acked Write 1 Sent Done Done false
13400 4274 Acked Flush 0 Sent Done Done false
13401 4275 Acked Write 7 Sent Done Done false
13402 4276 Acked Write 8 Sent Done Done false
13403 4277 Acked Read 3 Sent Done Done false
13404 4278 Acked Write 7 Sent Done Done false
13405 4279 Acked Read 8 Sent Done Done false
13406 4280 Acked Read 9 Sent Done Done false
13407 4281 Acked Read 4 Sent Done Done false
13408 4282 Acked Write 7 Sent Done Done false
13409 4283 Acked Write 9 Sent Done Done false
13410 4284 Acked Read 8 Sent Done Done false
13411 4285 Acked Write 6 Sent Done Done false
13412 4286 Acked Write 7 Sent Done Done false
13413 4287 Acked Read 8 Sent Done Done false
13414 4288 Acked Read 6 Sent Done Done false
13415 4289 Acked Flush 0 Sent Done Done false
13416 4290 Acked Write 10 Sent Done Done false
13417 4291 Acked Read 8 Sent Done Done false
13418 4292 Acked Read 9 Sent Done Done false
13419 4293 Acked Flush 0 Sent Done Done false
13420 4294 Acked Write 4 Sent Done Done false
13421 4295 Acked Read 4 Sent Done Done false
13422 4296 Acked Read 8 Sent Done Done false
13423 4297 Acked Flush 0 Sent Done Done false
13424 4298 Acked Write 1 Sent Done Done false
13425 4299 Acked Flush 0 Sent Done Done false
13426 4300 Acked Read 1 Sent Done Done false
13427 4301 Acked Write 10 Sent Done Done false
13428 4302 Acked Read 6 Sent Done Done false
13429 4303 Acked Read 3 Sent Done Done false
13430 4304 Acked Read 9 Sent Done Done false
13431 4305 Acked Read 3 Sent Done Done false
13432 4306 Acked Read 10 Sent Done Done false
13433 4307 Acked Read 10 Sent Done Done false
13434 4308 Acked Read 6 Sent Done Done false
13435 4309 Acked Read 1 Sent Done Done false
13436 4310 Acked Read 1 Sent Done Done false
13437 4311 Acked Read 9 Sent Done Done false
13438 4312 Acked Write 5 Sent Done Done false
13439 4313 Acked Flush 0 Sent Done Done false
13440 4314 Acked Read 9 Sent Done Done false
13441 4315 Acked Read 9 Sent Done Done false
13442 4316 Acked Flush 0 Sent Done Done false
13443 4317 Acked Write 6 Sent Done Done false
13444 4318 Acked Write 5 Sent Done Done false
13445 4319 Acked Write 6 Sent Done Done false
13446 4320 Acked Flush 0 Sent Done Done false
13447 4321 Acked Write 4 Sent Done Done false
13448 4322 Acked Write 7 Sent Done Done false
13449 4323 Acked Read 6 Sent Done Done false
13450 4324 Acked Flush 0 Sent Done Done false
13451 4325 Acked Write 4 Sent Done Done false
13452 4326 Acked Flush 0 Sent Done Done false
13453 STATES DS:0 DS:1 DS:2 TOTAL
13454 Sent 302 0 0 302
13455 Done 0 302 302 604
13456 Skipped 0 0 0 0
13457 Error 0 0 0 0
13458 Last Flush: 4024 4326 4326
13459 Downstairs last five completed: 4024 4023 4022 4021 4020
13460 Upstairs last five completed: 4326 4325 4324 4323 4322
13461 CLIENT: Up:0 ds:302 act:2
134622024-11-07T14:06:06.763ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134632024-11-07T14:06:06.763ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40548) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134642024-11-07T14:06:06.764ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134652024-11-07T14:06:06.765ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134662024-11-07T14:06:06.765ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134672024-11-07T14:06:06.765ZINFOcrucible: send last flush ID to this DS: 4024 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134682024-11-07T14:06:06.765ZINFOcrucible: Replied this last flush ID: 4024 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134692024-11-07T14:06:06.765ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134702024-11-07T14:06:06.765ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134712024-11-07T14:06:06.765ZINFOcrucible: [0] client re-new 302 jobs since flush 4024 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134722024-11-07T14:06:06.765ZINFOcrucible: [0] Replayed 302 jobs since flush: 4024 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
13473 ----------------------------------------------------------------
13474 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13475 Downstairs last five completed: 4326 4325 4324 4323 4322
13476 Upstairs last five completed: 4326 4325 4324 4323 4322
13477 CLIENT: Up:0 ds:0 act:3
13478 Replay: All jobs finished
13479 Main task has work to do, go find it
13480 got dsc Stop(0)
13481 stop 0
13482 Nov 07 14:06:09.067 INFO request completed, latency_us: 360, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 08bdba7a-7343-4144-a0d9-760828848e11, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
13483 [8810] Got stop action so:false kr:false
134842024-11-07T14:06:09.068ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134852024-11-07T14:06:09.068ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
134862024-11-07T14:06:09.068ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
13487 Nov 07 14:06:09.068{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:06:09.068653574Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"client":"0","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
134882024-11-07T14:06:09.068ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
13489 INFO request completed, latency_us: 137, response_code: 200, uri: /state/cid/0, method: GET, req_id: 49753b50-8184-4158-a826-7ac5d335eae5, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
134902024-11-07T14:06:09.068ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
13491 [8810] Exited with: ExitStatus(unix_wait_status(9))
13492 [8810][0] reports Exit
13493 Nov 07 14:06:13.070 INFO request completed, latency_us: 235, response_code: 200, uri: /state/cid/0, method: GET, req_id: da3f8e94-c297-4d36-b156-38973cad7a6a, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
13494 001/300 Write block 2898 len 28672 data: 2 2 2 3 3 3 3
13495 002/300 Write block 1901 len 16384 data: 2 3 3 3
13496 003/300 Read block 2185 len 28672
13497 004/300 Read block 4194 len 28672
13498 005/300 Read block 1088 len 36864
13499 006/300 Write block 736 len 16384 data: 4 4 3 3
13500 007/300 Write block 2534 len 40960 data: 2 2 2 2 2 6 6 6 5 5
13501 008/300 Read block 3343 len 36864
13502 009/300 Write block 2206 len 36864 data: 3 3 3 3 2 2 2 2 2
13503 010/300 Read block 2079 len 12288
13504 011/300 Write block 2375 len 24576 data: 2 2 2 2 2 3
13505 012/300 Read block 1490 len 12288
13506 013/300 Write block 362 len 20480 data: 3 3 2 2 2
13507 014/300 Read block 4773 len 40960
13508 015/300 Write block 3213 len 4096 data: 3
13509 016/300 Read block 661 len 16384
13510 017/300 Write block 3748 len 28672 data: 3 3 3 4 4 5 4
13511 018/300 Flush
13512 019/300 Write block 3142 len 8192 data: 3 2
13513 020/300 Read block 1802 len 8192
13514 021/300 Read block 1980 len 16384
13515 022/300 Write block 1843 len 20480 data: 5 4 4 4 4
13516 023/300 Read block 4687 len 36864
13517 024/300 Write block 144 len 24576 data: 2 2 3 4 4 4
13518 025/300 Read block 3571 len 12288
13519 026/300 Write block 435 len 16384 data: 3 4 4 6
13520 027/300 Write block 1325 len 4096 data: 4
13521 028/300 Read block 1159 len 40960
13522 029/300 Read block 3053 len 24576
13523 030/300 Write block 2272 len 36864 data: 4 4 4 4 4 5 3 3 3
13524 031/300 Read block 2639 len 36864
13525 032/300 Read block 397 len 40960
13526 033/300 Read block 3194 len 16384
13527 034/300 Read block 628 len 12288
13528 035/300 Read block 4342 len 20480
13529 036/300 Read block 992 len 20480
13530 037/300 Read block 2984 len 20480
13531 038/300 Read block 4884 len 28672
13532 039/300 Flush
13533 040/300 Write block 3303 len 4096 data: 3
13534 041/300 Flush
13535 042/300 Read block 449 len 36864
13536 043/300 Flush
13537 044/300 Read block 3446 len 40960
13538 045/300 Read block 158 len 4096
13539 046/300 Write block 4239 len 20480 data: 4 3 3 2 2
13540 047/300 Write block 2642 len 40960 data: 4 4 3 3 3 2 3 3 3 3
13541 048/300 Flush
13542 049/300 Write block 2080 len 16384 data: 3 3 3 4
13543 050/300 Flush
13544 051/300 Flush
13545 052/300 Read block 3503 len 16384
13546 053/300 Write block 230 len 24576 data: 3 3 3 3 3 3
13547 054/300 Write block 1993 len 36864 data: 2 2 2 2 3 3 3 3 3
13548 055/300 Write block 3542 len 20480 data: 2 2 2 2 2
13549 056/300 Flush
13550 057/300 Write block 4225 len 24576 data: 3 3 4 4 4 3
13551 058/300 Read block 3723 len 20480
13552 059/300 Read block 3075 len 28672
13553 060/300 Read block 1727 len 12288
13554 061/300 Read block 4619 len 24576
13555 062/300 Write block 4390 len 8192 data: 5 5
13556 063/300 Write block 1824 len 20480 data: 3 3 2 3 4
13557 064/300 Read block 3292 len 8192
13558 065/300 Read block 2664 len 12288
13559 066/300 Write block 4134 len 32768 data: 3 3 3 3 3 3 3 3
13560 067/300 Read block 3655 len 40960
13561 068/300 Write block 1771 len 40960 data: 5 4 4 4 4 3 2 2 3 3
13562 069/300 Read block 664 len 32768
13563 070/300 Read block 1688 len 36864
13564 071/300 Write block 393 len 12288 data: 2 3 4
13565 072/300 Write block 1346 len 16384 data: 2 3 3 4
13566 073/300 Read block 3974 len 40960
13567 074/300 Read block 2030 len 20480
13568 075/300 Read block 550 len 20480
13569 076/300 Write block 1751 len 28672 data: 6 5 5 4 4 4 3
13570 077/300 Read block 600 len 24576
13571 078/300 Read block 4514 len 28672
13572 079/300 Read block 689 len 12288
13573 080/300 Read block 2447 len 36864
13574 081/300 Read block 4963 len 16384
13575 082/300 Write block 722 len 16384 data: 5 4 4 4
13576 083/300 Write block 3487 len 36864 data: 4 5 3 3 3 4 3 3 3
13577 084/300 Read block 4062 len 36864
13578 085/300 Read block 1587 len 40960
13579 086/300 Write block 20 len 40960 data: 5 5 4 4 4 5 5 5 5 4
13580 087/300 Write block 1187 len 36864 data: 3 4 4 4 5 5 5 5 5
13581 088/300 Read block 1354 len 16384
13582 089/300 Write block 654 len 40960 data: 4 4 4 2 2 2 2 2 2 2
13583 090/300 Read block 3674 len 32768
13584 091/300 Read block 2520 len 8192
13585 092/300 Read block 4013 len 32768
13586 093/300 Write block 3520 len 12288 data: 2 2 3
13587 094/300 Write block 4268 len 40960 data: 3 4 4 4 4 4 3 3 3 4
13588 095/300 Read block 1510 len 4096
13589 096/300 Read block 4244 len 28672
13590 097/300 Flush
13591 098/300 Write block 195 len 8192 data: 3 3
13592 099/300 Write block 3093 len 8192 data: 4 2
13593 100/300 Write block 4614 len 36864 data: 3 3 2 2 2 2 3 2 2
13594 101/300 Read block 3495 len 16384
13595 102/300 Read block 1302 len 16384
13596 103/300 Read block 3862 len 40960
13597 104/300 Read block 2243 len 4096
13598 105/300 Write block 678 len 28672 data: 4 4 4 3 3 4 4
13599 106/300 Read block 3955 len 36864
13600 107/300 Read block 2338 len 40960
13601 108/300 Write block 2124 len 8192 data: 2 2
13602 109/300 Write block 3714 len 20480 data: 3 3 3 4 4
13603 110/300 Read block 4236 len 40960
13604 111/300 Read block 3716 len 12288
13605 112/300 Read block 1019 len 20480
13606 113/300 Read block 4439 len 12288
13607 114/300 Read block 148 len 8192
13608 115/300 Read block 331 len 12288
13609 116/300 Write block 3240 len 32768 data: 6 5 5 4 4 4 4 4
13610 117/300 Flush
13611 118/300 Write block 1211 len 4096 data: 4
13612 119/300 Read block 3220 len 20480
13613 120/300 Read block 977 len 8192
13614 121/300 Write block 3269 len 24576 data: 4 4 4 5 6 5
13615 122/300 Flush
13616 123/300 Read block 474 len 8192
13617 124/300 Read block 4498 len 12288
13618 125/300 Write block 1333 len 32768 data: 5 4 5 5 5 4 2 2
13619 126/300 Write block 3337 len 24576 data: 4 4 4 4 3 4
13620 127/300 Write block 1659 len 20480 data: 4 5 5 4 4
13621 128/300 Write block 367 len 8192 data: 2 3
13622 129/300 Write block 4220 len 24576 data: 3 3 3 3 3 4
13623 130/300 Write block 2277 len 12288 data: 6 4 4
13624 131/300 Flush
13625 132/300 Read block 3270 len 32768
13626 133/300 Write block 3951 len 24576 data: 2 2 2 2 2 2
13627 134/300 Read block 4129 len 8192
13628 135/300 Flush
13629 136/300 Write block 3840 len 36864 data: 3 3 4 4 4 2 3 4 3
13630 137/300 Read block 401 len 40960
13631 138/300 Read block 598 len 12288
13632 139/300 Write block 2849 len 24576 data: 2 2 2 2 3 4
13633 140/300 Read block 3177 len 32768
13634 141/300 Read block 2987 len 20480
13635 142/300 Read block 2157 len 40960
13636 143/300 Write block 2963 len 8192 data: 2 3
13637 144/300 Write block 1370 len 32768 data: 3 3 3 3 5 4 4 3
13638 145/300 Read block 346 len 16384
13639 146/300 Write block 4479 len 36864 data: 5 5 5 5 4 4 4 3 3
13640 147/300 Read block 4260 len 16384
13641 148/300 Write block 2662 len 4096 data: 4
13642 149/300 Read block 1080 len 4096
13643 150/300 Write block 2856 len 16384 data: 4 4 4 4
13644 151/300 Write block 3515 len 32768 data: 2 2 2 2 2 3 3 4
13645 152/300 Read block 1603 len 24576
13646 153/300 Read block 2470 len 20480
13647 154/300 Read block 3671 len 40960
13648 155/300 Flush
13649 156/300 Read block 2714 len 24576
13650 157/300 Read block 675 len 20480
13651 158/300 Write block 2958 len 20480 data: 3 3 3 2 2
13652 159/300 Read block 3599 len 36864
13653 160/300 Write block 2359 len 36864 data: 3 3 3 3 3 3 4 3 3
13654 161/300 Read block 729 len 20480
13655 162/300 Read block 1966 len 32768
13656 163/300 Read block 3610 len 20480
13657 164/300 Read block 3593 len 20480
13658 165/300 Read block 324 len 24576
13659 166/300 Write block 894 len 28672 data: 5 5 6 4 5 6 5
13660 167/300 Read block 1515 len 4096
13661 168/300 Write block 896 len 36864 data: 7 5 6 7 6 5 4 6 6
13662 169/300 Read block 2909 len 28672
13663 170/300 Write block 410 len 32768 data: 3 3 2 2 2 3 3 3
13664 171/300 Write block 2814 len 28672 data: 2 2 2 3 3 3 3
13665 172/300 Write block 1587 len 32768 data: 4 4 4 3 3 3 3 2
13666 173/300 Read block 647 len 36864
13667 174/300 Read block 1486 len 40960
13668 175/300 Read block 3834 len 40960
13669 176/300 Write block 3637 len 32768 data: 2 2 2 2 2 2 2 3
13670 177/300 Write block 3784 len 40960 data: 3 3 3 3 3 3 3 3 4 4
13671 178/300 Read block 4476 len 40960
13672 179/300 Write block 2846 len 40960 data: 3 3 3 3 3 3 3 4 5 4
13673 180/300 Read block 2475 len 36864
13674 181/300 Write block 2311 len 36864 data: 2 2 2 2 2 3 3 4 3
13675 182/300 Read block 602 len 24576
13676 183/300 Write block 1358 len 8192 data: 7 8
13677 184/300 Read block 268 len 4096
13678 185/300 Read block 3573 len 28672
13679 186/300 Read block 4957 len 12288
13680 187/300 Write block 3999 len 36864 data: 3 2 2 2 2 2 2 3 3
13681 188/300 Flush
13682 189/300 Read block 1980 len 4096
13683 190/300 Write block 2249 len 4096 data: 3
13684 191/300 Read block 1361 len 40960
13685 192/300 Write block 234 len 4096 data: 4
13686 193/300 Write block 1407 len 16384 data: 3 3 3 3
13687 194/300 Read block 476 len 40960
13688 195/300 Write block 147 len 32768 data: 5 5 5 3 4 4 4 5
13689 196/300 Write block 2101 len 32768 data: 2 2 2 3 2 3 3 2
13690 197/300 Flush
13691 198/300 Write block 3999 len 24576 data: 4 3 3 3 3 3
13692 199/300 Read block 1487 len 36864
13693 200/300 Read block 2452 len 12288
13694 201/300 Read block 2261 len 20480
13695 202/300 Write block 494 len 16384 data: 2 2 2 3
13696 203/300 Write block 3330 len 20480 data: 3 4 4 4 4
13697 204/300 Write block 1445 len 12288 data: 5 5 5
13698 205/300 Read block 4231 len 16384
13699 206/300 Read block 707 len 32768
13700 207/300 Write block 3431 len 16384 data: 4 4 4 2
13701 208/300 Read block 2618 len 32768
13702 209/300 Read block 2780 len 28672
13703 210/300 Read block 1254 len 20480
13704 211/300 Read block 3649 len 12288
13705 212/300 Write block 2852 len 12288 data: 4 5 6
13706 213/300 Flush
13707 214/300 Write block 3583 len 36864 data: 4 4 4 4 3 3 3 3 3
13708 215/300 Write block 810 len 8192 data: 2 2
13709 216/300 Write block 2725 len 40960 data: 4 4 3 3 2 3 3 3 3 3
13710 217/300 Write block 2588 len 40960 data: 2 2 3 3 4 4 4 4 4 4
13711 218/300 Flush
13712 219/300 Write block 3204 len 32768 data: 2 2 2 2 3 3 3 3
13713 220/300 Write block 1164 len 8192 data: 2 2
13714 221/300 Read block 456 len 20480
13715 222/300 Read block 2323 len 8192
13716 223/300 Read block 1149 len 8192
13717 224/300 Read block 20 len 8192
13718 225/300 Read block 508 len 8192
13719 226/300 Read block 1702 len 4096
13720 227/300 Flush
13721 228/300 Write block 1265 len 24576 data: 3 3 3 2 2 2
13722 229/300 Read block 1152 len 4096
13723 230/300 Flush
13724 231/300 Flush
13725 232/300 Read block 1080 len 8192
13726 233/300 Write block 4948 len 32768 data: 3 4 5 4 5 5 5 5
13727 234/300 Write block 4810 len 20480 data: 3 3 3 3 3
13728 235/300 Read block 278 len 28672
13729 236/300 Write block 4035 len 16384 data: 3 3 3 3
13730 237/300 Write block 3881 len 40960 data: 4 5 6 5 3 3 3 3 4 4
13731 238/300 Read block 3194 len 36864
13732 239/300 Write block 2399 len 8192 data: 3 3
13733 240/300 Read block 3146 len 32768
13734 241/300 Write block 1657 len 40960 data: 5 3 5 6 6 5 5 4 4 5
13735 242/300 Read block 2636 len 16384
13736 243/300 Write block 3373 len 24576 data: 2 2 4 5 5 4
13737 244/300 Write block 4295 len 20480 data: 5 4 4 5 5
13738 245/300 Write block 856 len 36864 data: 3 3 2 2 2 2 2 3 4
13739 246/300 Read block 3192 len 12288
13740 247/300 Write block 1213 len 40960 data: 4 4 3 3 3 3 3 2 2 3
13741 248/300 Write block 379 len 16384 data: 3 5 6 6
13742 249/300 Write block 3533 len 8192 data: 5 5
13743 250/300 Read block 4667 len 8192
13744 251/300 Read block 1732 len 4096
13745 252/300 Write block 2295 len 8192 data: 2 2
13746 253/300 Read block 2149 len 32768
13747 254/300 Write block 1071 len 8192 data: 3 4
13748 255/300 Read block 1762 len 8192
13749 256/300 Write block 226 len 8192 data: 4 4
13750 257/300 Write block 2076 len 28672 data: 3 3 3 3 4 4 4
13751 258/300 Write block 1134 len 24576 data: 3 2 2 2 4 4
13752 259/300 Write block 1129 len 40960 data: 3 3 2 2 2 4 3 3 3 5
13753 260/300 Read block 4621 len 8192
13754 261/300 Flush
13755 262/300 Flush
13756 263/300 Flush
13757 264/300 Read block 2965 len 12288
13758 265/300 Read block 4753 len 12288
13759 266/300 Write block 2731 len 8192 data: 4 4
13760 267/300 Write block 226 len 8192 data: 5 5
13761 268/300 Write block 667 len 36864 data: 2 2 2 3 2 2 2 2 2
13762 269/300 Read block 2736 len 36864
13763 270/300 Write block 4104 len 24576 data: 3 4 4 3 3 3
13764 271/300 Read block 956 len 32768
13765 272/300 Read block 1793 len 24576
13766 273/300 Flush
13767 274/300 Write block 444 len 24576 data: 4 3 2 3 3 3
13768 275/300 Read block 1210 len 24576
13769 276/300 Read block 3807 len 20480
13770 277/300 Write block 379 len 20480 data: 4 6 7 7 6
13771 278/300 Write block 286 len 36864 data: 2 3 3 3 3 2 4 5 5
13772 279/300 Read block 186 len 28672
13773 280/300 Read block 4725 len 36864
13774 281/300 Read block 4849 len 4096
13775 282/300 Read block 3932 len 12288
13776 283/300 Read block 3591 len 8192
13777 284/300 Write block 1726 len 8192 data: 3 2
13778 285/300 Write block 1943 len 32768 data: 4 5 5 5 4 4 4 4
13779 286/300 Write block 2481 len 20480 data: 5 6 6 4 5
13780 287/300 Write block 4198 len 36864 data: 4 4 4 4 4 4 5 3 3
13781 288/300 Read block 488 len 12288
13782 289/300 Read block 4025 len 36864
13783 290/300 Write block 2015 len 8192 data: 5 4
13784 291/300 Read block 2681 len 16384
13785 292/300 Read block 3755 len 20480
13786 293/300 Read block 1399 len 12288
13787 294/300 Flush
13788 295/300 Write block 2937 len 12288 data: 3 2 2
13789 296/300 Read block 1509 len 8192
13790 297/300 Read block 3377 len 16384
13791 298/300 Read block 3009 len 16384
13792 299/300 Read block 4247 len 12288
13793 300/300 Read block 3916 len 20480
13794 301/300 Read block 2717 len 4096
13795 Main task has work to do, go find it
13796 got dsc Start(0)
13797 start 0
13798 Nov 07 14:06:13.359 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/0, method: POST, req_id: 2143ce6f-882b-4721-9894-f5e75723e32b, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
13799 Replay: started 0, returned:Ok(())
13800 ----------------------------------------------------------------
13801 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13802 States: Offline Active Active
13803 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13804 4327 Acked Write 7 Sent Done Done false
13805 4328 Acked Write 4 Sent Done Done false
13806 4329 Acked Read 7 Sent Done Done false
13807 4330 Acked Read 7 Sent Done Done false
13808 4331 Acked Read 9 Sent Done Done false
13809 4332 Acked Write 4 Sent Done Done false
13810 4333 Acked Write 10 Sent Done Done false
13811 4334 Acked Read 9 Sent Done Done false
13812 4335 Acked Write 9 Sent Done Done false
13813 4336 Acked Read 3 Sent Done Done false
13814 4337 Acked Write 6 Sent Done Done false
13815 4338 Acked Read 3 Sent Done Done false
13816 4339 Acked Write 5 Sent Done Done false
13817 4340 Acked Read 10 Sent Done Done false
13818 4341 Acked Write 1 Sent Done Done false
13819 4342 Acked Read 4 Sent Done Done false
13820 4343 Acked Write 7 Sent Done Done false
13821 4344 Acked Flush 0 Sent Done Done false
13822 4345 Acked Write 2 Sent Done Done false
13823 4346 Acked Read 2 Sent Done Done false
13824 4347 Acked Read 4 Sent Done Done false
13825 4348 Acked Write 5 Sent Done Done false
13826 4349 Acked Read 9 Sent Done Done false
13827 4350 Acked Write 6 Sent Done Done false
13828 4351 Acked Read 3 Sent Done Done false
13829 4352 Acked Write 4 Sent Done Done false
13830 4353 Acked Write 1 Sent Done Done false
13831 4354 Acked Read 10 Sent Done Done false
13832 4355 Acked Read 6 Sent Done Done false
13833 4356 Acked Write 9 Sent Done Done false
13834 4357 Acked Read 9 Sent Done Done false
13835 4358 Acked Read 10 Sent Done Done false
13836 4359 Acked Read 4 Sent Done Done false
13837 4360 Acked Read 3 Sent Done Done false
13838 4361 Acked Read 5 Sent Done Done false
13839 4362 Acked Read 5 Sent Done Done false
13840 4363 Acked Read 5 Sent Done Done false
13841 4364 Acked Read 7 Sent Done Done false
13842 4365 Acked Flush 0 Sent Done Done false
13843 4366 Acked Write 1 Sent Done Done false
13844 4367 Acked Flush 0 Sent Done Done false
13845 4368 Acked Read 9 Sent Done Done false
13846 4369 Acked Flush 0 Sent Done Done false
13847 4370 Acked Read 10 Sent Done Done false
13848 4371 Acked Read 1 Sent Done Done false
13849 4372 Acked Write 5 Sent Done Done false
13850 4373 Acked Write 10 Sent Done Done false
13851 4374 Acked Flush 0 Sent Done Done false
13852 4375 Acked Write 4 Sent Done Done false
13853 4376 Acked Flush 0 Sent Done Done false
13854 4377 Acked Flush 0 Sent Done Done false
13855 4378 Acked Read 4 Sent Done Done false
13856 4379 Acked Write 6 Sent Done Done false
13857 4380 Acked Write 9 Sent Done Done false
13858 4381 Acked Write 5 Sent Done Done false
13859 4382 Acked Flush 0 Sent Done Done false
13860 4383 Acked Write 6 Sent Done Done false
13861 4384 Acked Read 5 Sent Done Done false
13862 4385 Acked Read 7 Sent Done Done false
13863 4386 Acked Read 3 Sent Done Done false
13864 4387 Acked Read 6 Sent Done Done false
13865 4388 Acked Write 2 Sent Done Done false
13866 4389 Acked Write 5 Sent Done Done false
13867 4390 Acked Read 2 Sent Done Done false
13868 4391 Acked Read 3 Sent Done Done false
13869 4392 Acked Write 8 Sent Done Done false
13870 4393 Acked Read 10 Sent Done Done false
13871 4394 Acked Write 10 Sent Done Done false
13872 4395 Acked Read 8 Sent Done Done false
13873 4396 Acked Read 9 Sent Done Done false
13874 4397 Acked Write 3 Sent Done Done false
13875 4398 Acked Write 4 Sent Done Done false
13876 4399 Acked Read 10 Sent Done Done false
13877 4400 Acked Read 5 Sent Done Done false
13878 4401 Acked Read 5 Sent Done Done false
13879 4402 Acked Write 7 Sent Done Done false
13880 4403 Acked Read 6 Sent Done Done false
13881 4404 Acked Read 7 Sent Done Done false
13882 4405 Acked Read 3 Sent Done Done false
13883 4406 Acked Read 9 Sent Done Done false
13884 4407 Acked Read 4 Sent Done Done false
13885 4408 Acked Write 4 Sent Done Done false
13886 4409 Acked Write 9 Sent Done Done false
13887 4410 Acked Read 9 Sent Done Done false
13888 4411 Acked Read 10 Sent Done Done false
13889 4412 Acked Write 10 Sent Done Done false
13890 4413 Acked Write 9 Sent Done Done false
13891 4414 Acked Read 4 Sent Done Done false
13892 4415 Acked Write 10 Sent Done Done false
13893 4416 Acked Read 8 Sent Done Done false
13894 4417 Acked Read 2 Sent Done Done false
13895 4418 Acked Read 8 Sent Done Done false
13896 4419 Acked Write 3 Sent Done Done false
13897 4420 Acked Write 10 Sent Done Done false
13898 4421 Acked Read 1 Sent Done Done false
13899 4422 Acked Read 7 Sent Done Done false
13900 4423 Acked Flush 0 Sent Done Done false
13901 4424 Acked Write 2 Sent Done Done false
13902 4425 Acked Write 2 Sent Done Done false
13903 4426 Acked Write 9 Sent Done Done false
13904 4427 Acked Read 4 Sent Done Done false
13905 4428 Acked Read 4 Sent Done Done false
13906 4429 Acked Read 10 Sent Done Done false
13907 4430 Acked Read 1 Sent Done Done false
13908 4431 Acked Write 7 Sent Done Done false
13909 4432 Acked Read 9 Sent Done Done false
13910 4433 Acked Read 10 Sent Done Done false
13911 4434 Acked Write 2 Sent Done Done false
13912 4435 Acked Write 5 Sent Done Done false
13913 4436 Acked Read 10 Sent Done Done false
13914 4437 Acked Read 3 Sent Done Done false
13915 4438 Acked Read 5 Sent Done Done false
13916 4439 Acked Read 3 Sent Done Done false
13917 4440 Acked Read 2 Sent Done Done false
13918 4441 Acked Read 3 Sent Done Done false
13919 4442 Acked Write 8 Sent Done Done false
13920 4443 Acked Flush 0 Sent Done Done false
13921 4444 Acked Write 1 Sent Done Done false
13922 4445 Acked Read 5 Sent Done Done false
13923 4446 Acked Read 2 Sent Done Done false
13924 4447 Acked Write 6 Sent Done Done false
13925 4448 Acked Flush 0 Sent Done Done false
13926 4449 Acked Read 2 Sent Done Done false
13927 4450 Acked Read 3 Sent Done Done false
13928 4451 Acked Write 8 Sent Done Done false
13929 4452 Acked Write 6 Sent Done Done false
13930 4453 Acked Write 5 Sent Done Done false
13931 4454 Acked Write 2 Sent Done Done false
13932 4455 Acked Write 6 Sent Done Done false
13933 4456 Acked Write 3 Sent Done Done false
13934 4457 Acked Flush 0 Sent Done Done false
13935 4458 Acked Read 8 Sent Done Done false
13936 4459 Acked Write 6 Sent Done Done false
13937 4460 Acked Read 2 Sent Done Done false
13938 4461 Acked Flush 0 Sent Done Done false
13939 4462 Acked Write 9 Sent Done Done false
13940 4463 Acked Read 10 Sent Done Done false
13941 4464 Acked Read 3 Sent Done Done false
13942 4465 Acked Write 6 Sent Done Done false
13943 4466 Acked Read 8 Sent Done Done false
13944 4467 Acked Read 5 Sent Done Done false
13945 4468 Acked Read 10 Sent Done Done false
13946 4469 Acked Write 2 Sent Done Done false
13947 4470 Acked Write 8 Sent Done Done false
13948 4471 Acked Read 4 Sent Done Done false
13949 4472 Acked Write 9 Sent Done Done false
13950 4473 Acked Read 4 Sent Done Done false
13951 4474 Acked Write 1 Sent Done Done false
13952 4475 Acked Read 1 Sent Done Done false
13953 4476 Acked Write 4 Sent Done Done false
13954 4477 Acked Write 8 Sent Done Done false
13955 4478 Acked Read 6 Sent Done Done false
13956 4479 Acked Read 5 Sent Done Done false
13957 4480 Acked Read 10 Sent Done Done false
13958 4481 Acked Flush 0 Sent Done Done false
13959 4482 Acked Read 6 Sent Done Done false
13960 4483 Acked Read 5 Sent Done Done false
13961 4484 Acked Write 5 Sent Done Done false
13962 4485 Acked Read 9 Sent Done Done false
13963 4486 Acked Write 9 Sent Done Done false
13964 4487 Acked Read 5 Sent Done Done false
13965 4488 Acked Read 8 Sent Done Done false
13966 4489 Acked Read 5 Sent Done Done false
13967 4490 Acked Read 5 Sent Done Done false
13968 4491 Acked Read 6 Sent Done Done false
13969 4492 Acked Write 7 Sent Done Done false
13970 4493 Acked Read 1 Sent Done Done false
13971 4494 Acked Write 9 Sent Done Done false
13972 4495 Acked Read 7 Sent Done Done false
13973 4496 Acked Write 8 Sent Done Done false
13974 4497 Acked Write 7 Sent Done Done false
13975 4498 Acked Write 8 Sent Done Done false
13976 4499 Acked Read 9 Sent Done Done false
13977 4500 Acked Read 10 Sent Done Done false
13978 4501 Acked Read 10 Sent Done Done false
13979 4502 Acked Write 8 Sent Done Done false
13980 4503 Acked Write 10 Sent Done Done false
13981 4504 Acked Read 10 Sent Done Done false
13982 4505 Acked Write 10 Sent Done Done false
13983 4506 Acked Read 9 Sent Done Done false
13984 4507 Acked Write 9 Sent Done Done false
13985 4508 Acked Read 6 Sent Done Done false
13986 4509 Acked Write 2 Sent Done Done false
13987 4510 Acked Read 1 Sent Done Done false
13988 4511 Acked Read 7 Sent Done Done false
13989 4512 Acked Read 3 Sent Done Done false
13990 4513 Acked Write 9 Sent Done Done false
13991 4514 Acked Flush 0 Sent Done Done false
13992 4515 Acked Read 1 Sent Done Done false
13993 4516 Acked Write 1 Sent Done Done false
13994 4517 Acked Read 10 Sent Done Done false
13995 4518 Acked Write 1 Sent Done Done false
13996 4519 Acked Write 4 Sent Done Done false
13997 4520 Acked Read 10 Sent Done Done false
13998 4521 Acked Write 8 Sent Done Done false
13999 4522 Acked Write 8 Sent Done Done false
14000 4523 Acked Flush 0 Sent Done Done false
14001 4524 Acked Write 6 Sent Done Done false
14002 4525 Acked Read 9 Sent Done Done false
14003 4526 Acked Read 3 Sent Done Done false
14004 4527 Acked Read 5 Sent Done Done false
14005 4528 Acked Write 4 Sent Done Done false
14006 4529 Acked Write 5 Sent Done Done false
14007 4530 Acked Write 3 Sent Done Done false
14008 4531 Acked Read 4 Sent Done Done false
14009 4532 Acked Read 8 Sent Done Done false
14010 4533 Acked Write 4 Sent Done Done false
14011 4534 Acked Read 8 Sent Done Done false
14012 4535 Acked Read 7 Sent Done Done false
14013 4536 Acked Read 5 Sent Done Done false
14014 4537 Acked Read 3 Sent Done Done false
14015 4538 Acked Write 3 Sent Done Done false
14016 4539 Acked Flush 0 Sent Done Done false
14017 4540 Acked Write 9 Sent Done Done false
14018 4541 Acked Write 2 Sent Done Done false
14019 4542 Acked Write 10 Sent Done Done false
14020 4543 Acked Write 10 Sent Done Done false
14021 4544 Acked Flush 0 Sent Done Done false
14022 4545 Acked Write 8 Sent Done Done false
14023 4546 Acked Write 2 Sent Done Done false
14024 4547 Acked Read 5 Sent Done Done false
14025 4548 Acked Read 2 Sent Done Done false
14026 4549 Acked Read 2 Sent Done Done false
14027 4550 Acked Read 2 Sent Done Done false
14028 4551 Acked Read 2 Sent Done Done false
14029 4552 Acked Read 1 Sent Done Done false
14030 4553 Acked Flush 0 Sent Done Done false
14031 4554 Acked Write 6 Sent Done Done false
14032 4555 Acked Read 1 Sent Done Done false
14033 4556 Acked Flush 0 Sent Done Done false
14034 4557 Acked Flush 0 Sent Done Done false
14035 4558 Acked Read 2 Sent Done Done false
14036 4559 Acked Write 8 Sent Done Done false
14037 4560 Acked Write 5 Sent Done Done false
14038 4561 Acked Read 7 Sent Done Done false
14039 4562 Acked Write 4 Sent Done Done false
14040 4563 Acked Write 10 Sent Done Done false
14041 4564 Acked Read 9 Sent Done Done false
14042 4565 Acked Write 2 Sent Done Done false
14043 4566 Acked Read 8 Sent Done Done false
14044 4567 Acked Write 10 Sent Done Done false
14045 4568 Acked Read 4 Sent Done Done false
14046 4569 Acked Write 6 Sent Done Done false
14047 4570 Acked Write 5 Sent Done Done false
14048 4571 Acked Write 9 Sent Done Done false
14049 4572 Acked Read 3 Sent Done Done false
14050 4573 Acked Write 10 Sent Done Done false
14051 4574 Acked Write 4 Sent Done Done false
14052 4575 Acked Write 2 Sent Done Done false
14053 4576 Acked Read 2 Sent Done Done false
14054 4577 Acked Read 1 Sent Done Done false
14055 4578 Acked Write 2 Sent Done Done false
14056 4579 Acked Read 8 Sent Done Done false
14057 4580 Acked Write 2 Sent Done Done false
14058 4581 Acked Read 2 Sent Done Done false
14059 4582 Acked Write 2 Sent Done Done false
14060 4583 Acked Write 7 Sent Done Done false
14061 4584 Acked Write 6 Sent Done Done false
14062 4585 Acked Write 10 Sent Done Done false
14063 4586 Acked Read 2 Sent Done Done false
14064 4587 Acked Flush 0 Sent Done Done false
14065 4588 Acked Flush 0 Sent Done Done false
14066 4589 Acked Flush 0 Sent Done Done false
14067 4590 Acked Read 3 Sent Done Done false
14068 4591 Acked Read 3 Sent Done Done false
14069 4592 Acked Write 2 Sent Done Done false
14070 4593 Acked Write 2 Sent Done Done false
14071 4594 Acked Write 9 Sent Done Done false
14072 4595 Acked Read 9 Sent Done Done false
14073 4596 Acked Write 6 Sent Done Done false
14074 4597 Acked Read 8 Sent Done Done false
14075 4598 Acked Read 6 Sent Done Done false
14076 4599 Acked Flush 0 Sent Done Done false
14077 4600 Acked Write 6 Sent Done Done false
14078 4601 Acked Read 6 Sent Done Done false
14079 4602 Acked Read 5 Sent Done Done false
14080 4603 Acked Write 5 Sent Done Done false
14081 4604 Acked Write 9 Sent Done Done false
14082 4605 Acked Read 7 Sent Done Done false
14083 4606 Acked Read 9 Sent Done Done false
14084 4607 Acked Read 1 Sent Done Done false
14085 4608 Acked Read 3 Sent Done Done false
14086 4609 Acked Read 2 Sent Done Done false
14087 4610 Acked Write 2 Sent Done Done false
14088 4611 Acked Write 8 Sent Done Done false
14089 4612 Acked Write 5 Sent Done Done false
14090 4613 Acked Write 9 Sent Done Done false
14091 4614 Acked Read 3 Sent Done Done false
14092 4615 Acked Read 9 Sent Done Done false
14093 4616 Acked Write 2 Sent Done Done false
14094 4617 Acked Read 4 Sent Done Done false
14095 4618 Acked Read 5 Sent Done Done false
14096 4619 Acked Read 3 Sent Done Done false
14097 4620 Acked Flush 0 Sent Done Done false
14098 4621 Acked Write 3 Sent Done Done false
14099 4622 Acked Read 2 Sent Done Done false
14100 4623 Acked Read 4 Sent Done Done false
14101 4624 Acked Read 4 Sent Done Done false
14102 4625 Acked Read 3 Sent Done Done false
14103 4626 Acked Read 5 Sent Done Done false
14104 4627 Acked Read 1 Sent Done Done false
14105 STATES DS:0 DS:1 DS:2 TOTAL
14106 Sent 301 0 0 301
14107 Done 0 301 301 602
14108 Skipped 0 0 0 0
14109 Error 0 0 0 0
14110 Last Flush: 4326 4620 4620
14111 Downstairs last five completed: 4326 4325 4324 4323 4322
14112 Upstairs last five completed: 4627 4626 4625 4624 4623
14113 CLIENT: Up:0 ds:301 act:2
14114 [8810] got start action
14115 [8810] I am going to restart
14116 Starting downstairs at port 8810
14117 Make output file at "/tmp/dsc/downstairs-8810.txt"
14118 [8810][0] reports Starting
14119 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1211)
14120 [8810][0] reports Running
14121 ----------------------------------------------------------------
14122 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14123 States: Offline Active Active
14124 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14125 4327 Acked Write 7 Sent Done Done false
14126 4328 Acked Write 4 Sent Done Done false
14127 4329 Acked Read 7 Sent Done Done false
14128 4330 Acked Read 7 Sent Done Done false
14129 4331 Acked Read 9 Sent Done Done false
14130 4332 Acked Write 4 Sent Done Done false
14131 4333 Acked Write 10 Sent Done Done false
14132 4334 Acked Read 9 Sent Done Done false
14133 4335 Acked Write 9 Sent Done Done false
14134 4336 Acked Read 3 Sent Done Done false
14135 4337 Acked Write 6 Sent Done Done false
14136 4338 Acked Read 3 Sent Done Done false
14137 4339 Acked Write 5 Sent Done Done false
14138 4340 Acked Read 10 Sent Done Done false
14139 4341 Acked Write 1 Sent Done Done false
14140 4342 Acked Read 4 Sent Done Done false
14141 4343 Acked Write 7 Sent Done Done false
14142 4344 Acked Flush 0 Sent Done Done false
14143 4345 Acked Write 2 Sent Done Done false
14144 4346 Acked Read 2 Sent Done Done false
14145 4347 Acked Read 4 Sent Done Done false
14146 4348 Acked Write 5 Sent Done Done false
14147 4349 Acked Read 9 Sent Done Done false
14148 4350 Acked Write 6 Sent Done Done false
14149 4351 Acked Read 3 Sent Done Done false
14150 4352 Acked Write 4 Sent Done Done false
14151 4353 Acked Write 1 Sent Done Done false
14152 4354 Acked Read 10 Sent Done Done false
14153 4355 Acked Read 6 Sent Done Done false
14154 4356 Acked Write 9 Sent Done Done false
14155 4357 Acked Read 9 Sent Done Done false
14156 4358 Acked Read 10 Sent Done Done false
14157 4359 Acked Read 4 Sent Done Done false
14158 4360 Acked Read 3 Sent Done Done false
14159 4361 Acked Read 5 Sent Done Done false
14160 4362 Acked Read 5 Sent Done Done false
14161 4363 Acked Read 5 Sent Done Done false
14162 4364 Acked Read 7 Sent Done Done false
14163 4365 Acked Flush 0 Sent Done Done false
14164 4366 Acked Write 1 Sent Done Done false
14165 4367 Acked Flush 0 Sent Done Done false
14166 4368 Acked Read 9 Sent Done Done false
14167 4369 Acked Flush 0 Sent Done Done false
14168 4370 Acked Read 10 Sent Done Done false
14169 4371 Acked Read 1 Sent Done Done false
14170 4372 Acked Write 5 Sent Done Done false
14171 4373 Acked Write 10 Sent Done Done false
14172 4374 Acked Flush 0 Sent Done Done false
14173 4375 Acked Write 4 Sent Done Done false
14174 4376 Acked Flush 0 Sent Done Done false
14175 4377 Acked Flush 0 Sent Done Done false
14176 4378 Acked Read 4 Sent Done Done false
14177 4379 Acked Write 6 Sent Done Done false
14178 4380 Acked Write 9 Sent Done Done false
14179 4381 Acked Write 5 Sent Done Done false
14180 4382 Acked Flush 0 Sent Done Done false
14181 4383 Acked Write 6 Sent Done Done false
14182 4384 Acked Read 5 Sent Done Done false
14183 4385 Acked Read 7 Sent Done Done false
14184 4386 Acked Read 3 Sent Done Done false
14185 4387 Acked Read 6 Sent Done Done false
14186 4388 Acked Write 2 Sent Done Done false
14187 4389 Acked Write 5 Sent Done Done false
14188 4390 Acked Read 2 Sent Done Done false
14189 4391 Acked Read 3 Sent Done Done false
14190 4392 Acked Write 8 Sent Done Done false
14191 4393 Acked Read 10 Sent Done Done false
14192 4394 Acked Write 10 Sent Done Done false
14193 4395 Acked Read 8 Sent Done Done false
14194 4396 Acked Read 9 Sent Done Done false
14195 4397 Acked Write 3 Sent Done Done false
14196 4398 Acked Write 4 Sent Done Done false
14197 4399 Acked Read 10 Sent Done Done false
14198 4400 Acked Read 5 Sent Done Done false
14199 4401 Acked Read 5 Sent Done Done false
14200 4402 Acked Write 7 Sent Done Done false
14201 4403 Acked Read 6 Sent Done Done false
14202 4404 Acked Read 7 Sent Done Done false
14203 4405 Acked Read 3 Sent Done Done false
14204 4406 Acked Read 9 Sent Done Done false
14205 4407 Acked Read 4 Sent Done Done false
14206 4408 Acked Write 4 Sent Done Done false
14207 4409 Acked Write 9 Sent Done Done false
14208 4410 Acked Read 9 Sent Done Done false
14209 4411 Acked Read 10 Sent Done Done false
14210 4412 Acked Write 10 Sent Done Done false
14211 4413 Acked Write 9 Sent Done Done false
14212 4414 Acked Read 4 Sent Done Done false
14213 4415 Acked Write 10 Sent Done Done false
14214 4416 Acked Read 8 Sent Done Done false
14215 4417 Acked Read 2 Sent Done Done false
14216 4418 Acked Read 8 Sent Done Done false
14217 4419 Acked Write 3 Sent Done Done false
14218 4420 Acked Write 10 Sent Done Done false
14219 4421 Acked Read 1 Sent Done Done false
14220 4422 Acked Read 7 Sent Done Done false
14221 4423 Acked Flush 0 Sent Done Done false
14222 4424 Acked Write 2 Sent Done Done false
14223 4425 Acked Write 2 Sent Done Done false
14224 4426 Acked Write 9 Sent Done Done false
14225 4427 Acked Read 4 Sent Done Done false
14226 4428 Acked Read 4 Sent Done Done false
14227 4429 Acked Read 10 Sent Done Done false
14228 4430 Acked Read 1 Sent Done Done false
14229 4431 Acked Write 7 Sent Done Done false
14230 4432 Acked Read 9 Sent Done Done false
14231 4433 Acked Read 10 Sent Done Done false
14232 4434 Acked Write 2 Sent Done Done false
14233 4435 Acked Write 5 Sent Done Done false
14234 4436 Acked Read 10 Sent Done Done false
14235 4437 Acked Read 3 Sent Done Done false
14236 4438 Acked Read 5 Sent Done Done false
14237 4439 Acked Read 3 Sent Done Done false
14238 4440 Acked Read 2 Sent Done Done false
14239 4441 Acked Read 3 Sent Done Done false
14240 4442 Acked Write 8 Sent Done Done false
14241 4443 Acked Flush 0 Sent Done Done false
14242 4444 Acked Write 1 Sent Done Done false
14243 4445 Acked Read 5 Sent Done Done false
14244 4446 Acked Read 2 Sent Done Done false
14245 4447 Acked Write 6 Sent Done Done false
14246 4448 Acked Flush 0 Sent Done Done false
14247 4449 Acked Read 2 Sent Done Done false
14248 4450 Acked Read 3 Sent Done Done false
14249 4451 Acked Write 8 Sent Done Done false
14250 4452 Acked Write 6 Sent Done Done false
14251 4453 Acked Write 5 Sent Done Done false
14252 4454 Acked Write 2 Sent Done Done false
14253 4455 Acked Write 6 Sent Done Done false
14254 4456 Acked Write 3 Sent Done Done false
14255 4457 Acked Flush 0 Sent Done Done false
14256 4458 Acked Read 8 Sent Done Done false
14257 4459 Acked Write 6 Sent Done Done false
14258 4460 Acked Read 2 Sent Done Done false
14259 4461 Acked Flush 0 Sent Done Done false
14260 4462 Acked Write 9 Sent Done Done false
14261 4463 Acked Read 10 Sent Done Done false
14262 4464 Acked Read 3 Sent Done Done false
14263 4465 Acked Write 6 Sent Done Done false
14264 4466 Acked Read 8 Sent Done Done false
14265 4467 Acked Read 5 Sent Done Done false
14266 4468 Acked Read 10 Sent Done Done false
14267 4469 Acked Write 2 Sent Done Done false
14268 4470 Acked Write 8 Sent Done Done false
14269 4471 Acked Read 4 Sent Done Done false
14270 4472 Acked Write 9 Sent Done Done false
14271 4473 Acked Read 4 Sent Done Done false
14272 4474 Acked Write 1 Sent Done Done false
14273 4475 Acked Read 1 Sent Done Done false
14274 4476 Acked Write 4 Sent Done Done false
14275 4477 Acked Write 8 Sent Done Done false
14276 4478 Acked Read 6 Sent Done Done false
14277 4479 Acked Read 5 Sent Done Done false
14278 4480 Acked Read 10 Sent Done Done false
14279 4481 Acked Flush 0 Sent Done Done false
14280 4482 Acked Read 6 Sent Done Done false
14281 4483 Acked Read 5 Sent Done Done false
14282 4484 Acked Write 5 Sent Done Done false
14283 4485 Acked Read 9 Sent Done Done false
14284 4486 Acked Write 9 Sent Done Done false
14285 4487 Acked Read 5 Sent Done Done false
14286 4488 Acked Read 8 Sent Done Done false
14287 4489 Acked Read 5 Sent Done Done false
14288 4490 Acked Read 5 Sent Done Done false
14289 4491 Acked Read 6 Sent Done Done false
14290 4492 Acked Write 7 Sent Done Done false
14291 4493 Acked Read 1 Sent Done Done false
14292 4494 Acked Write 9 Sent Done Done false
14293 4495 Acked Read 7 Sent Done Done false
14294 4496 Acked Write 8 Sent Done Done false
14295 4497 Acked Write 7 Sent Done Done false
14296 4498 Acked Write 8 Sent Done Done false
14297 4499 Acked Read 9 Sent Done Done false
14298 4500 Acked Read 10 Sent Done Done false
14299 4501 Acked Read 10 Sent Done Done false
14300 4502 Acked Write 8 Sent Done Done false
14301 4503 Acked Write 10 Sent Done Done false
14302 4504 Acked Read 10 Sent Done Done false
14303 4505 Acked Write 10 Sent Done Done false
14304 4506 Acked Read 9 Sent Done Done false
14305 4507 Acked Write 9 Sent Done Done false
14306 4508 Acked Read 6 Sent Done Done false
14307 4509 Acked Write 2 Sent Done Done false
14308 4510 Acked Read 1 Sent Done Done false
14309 4511 Acked Read 7 Sent Done Done false
14310 4512 Acked Read 3 Sent Done Done false
14311 4513 Acked Write 9 Sent Done Done false
14312 4514 Acked Flush 0 Sent Done Done false
14313 4515 Acked Read 1 Sent Done Done false
14314 4516 Acked Write 1 Sent Done Done false
14315 4517 Acked Read 10 Sent Done Done false
14316 4518 Acked Write 1 Sent Done Done false
14317 4519 Acked Write 4 Sent Done Done false
14318 4520 Acked Read 10 Sent Done Done false
14319 4521 Acked Write 8 Sent Done Done false
14320 4522 Acked Write 8 Sent Done Done false
14321 4523 Acked Flush 0 Sent Done Done false
14322 4524 Acked Write 6 Sent Done Done false
14323 4525 Acked Read 9 Sent Done Done false
14324 4526 Acked Read 3 Sent Done Done false
14325 4527 Acked Read 5 Sent Done Done false
14326 4528 Acked Write 4 Sent Done Done false
14327 4529 Acked Write 5 Sent Done Done false
14328 4530 Acked Write 3 Sent Done Done false
14329 4531 Acked Read 4 Sent Done Done false
14330 4532 Acked Read 8 Sent Done Done false
14331 4533 Acked Write 4 Sent Done Done false
14332 4534 Acked Read 8 Sent Done Done false
14333 4535 Acked Read 7 Sent Done Done false
14334 4536 Acked Read 5 Sent Done Done false
14335 4537 Acked Read 3 Sent Done Done false
14336 4538 Acked Write 3 Sent Done Done false
14337 4539 Acked Flush 0 Sent Done Done false
14338 4540 Acked Write 9 Sent Done Done false
14339 4541 Acked Write 2 Sent Done Done false
14340 4542 Acked Write 10 Sent Done Done false
14341 4543 Acked Write 10 Sent Done Done false
14342 4544 Acked Flush 0 Sent Done Done false
14343 4545 Acked Write 8 Sent Done Done false
14344 4546 Acked Write 2 Sent Done Done false
14345 4547 Acked Read 5 Sent Done Done false
14346 4548 Acked Read 2 Sent Done Done false
14347 4549 Acked Read 2 Sent Done Done false
14348 4550 Acked Read 2 Sent Done Done false
14349 4551 Acked Read 2 Sent Done Done false
14350 4552 Acked Read 1 Sent Done Done false
14351 4553 Acked Flush 0 Sent Done Done false
14352 4554 Acked Write 6 Sent Done Done false
14353 4555 Acked Read 1 Sent Done Done false
14354 4556 Acked Flush 0 Sent Done Done false
14355 4557 Acked Flush 0 Sent Done Done false
14356 4558 Acked Read 2 Sent Done Done false
14357 4559 Acked Write 8 Sent Done Done false
14358 4560 Acked Write 5 Sent Done Done false
14359 4561 Acked Read 7 Sent Done Done false
14360 4562 Acked Write 4 Sent Done Done false
14361 4563 Acked Write 10 Sent Done Done false
14362 4564 Acked Read 9 Sent Done Done false
14363 4565 Acked Write 2 Sent Done Done false
14364 4566 Acked Read 8 Sent Done Done false
14365 4567 Acked Write 10 Sent Done Done false
14366 4568 Acked Read 4 Sent Done Done false
14367 4569 Acked Write 6 Sent Done Done false
14368 4570 Acked Write 5 Sent Done Done false
14369 4571 Acked Write 9 Sent Done Done false
14370 4572 Acked Read 3 Sent Done Done false
14371 4573 Acked Write 10 Sent Done Done false
14372 4574 Acked Write 4 Sent Done Done false
14373 4575 Acked Write 2 Sent Done Done false
14374 4576 Acked Read 2 Sent Done Done false
14375 4577 Acked Read 1 Sent Done Done false
14376 4578 Acked Write 2 Sent Done Done false
14377 4579 Acked Read 8 Sent Done Done false
14378 4580 Acked Write 2 Sent Done Done false
14379 4581 Acked Read 2 Sent Done Done false
14380 4582 Acked Write 2 Sent Done Done false
14381 4583 Acked Write 7 Sent Done Done false
14382 4584 Acked Write 6 Sent Done Done false
14383 4585 Acked Write 10 Sent Done Done false
14384 4586 Acked Read 2 Sent Done Done false
14385 4587 Acked Flush 0 Sent Done Done false
14386 4588 Acked Flush 0 Sent Done Done false
14387 4589 Acked Flush 0 Sent Done Done false
14388 4590 Acked Read 3 Sent Done Done false
14389 4591 Acked Read 3 Sent Done Done false
14390 4592 Acked Write 2 Sent Done Done false
14391 4593 Acked Write 2 Sent Done Done false
14392 4594 Acked Write 9 Sent Done Done false
14393 4595 Acked Read 9 Sent Done Done false
14394 4596 Acked Write 6 Sent Done Done false
14395 4597 Acked Read 8 Sent Done Done false
14396 4598 Acked Read 6 Sent Done Done false
14397 4599 Acked Flush 0 Sent Done Done false
14398 4600 Acked Write 6 Sent Done Done false
14399 4601 Acked Read 6 Sent Done Done false
14400 4602 Acked Read 5 Sent Done Done false
14401 4603 Acked Write 5 Sent Done Done false
14402 4604 Acked Write 9 Sent Done Done false
14403 4605 Acked Read 7 Sent Done Done false
14404 4606 Acked Read 9 Sent Done Done false
14405 4607 Acked Read 1 Sent Done Done false
14406 4608 Acked Read 3 Sent Done Done false
14407 4609 Acked Read 2 Sent Done Done false
14408 4610 Acked Write 2 Sent Done Done false
14409 4611 Acked Write 8 Sent Done Done false
14410 4612 Acked Write 5 Sent Done Done false
14411 4613 Acked Write 9 Sent Done Done false
14412 4614 Acked Read 3 Sent Done Done false
14413 4615 Acked Read 9 Sent Done Done false
14414 4616 Acked Write 2 Sent Done Done false
14415 4617 Acked Read 4 Sent Done Done false
14416 4618 Acked Read 5 Sent Done Done false
14417 4619 Acked Read 3 Sent Done Done false
14418 4620 Acked Flush 0 Sent Done Done false
14419 4621 Acked Write 3 Sent Done Done false
14420 4622 Acked Read 2 Sent Done Done false
14421 4623 Acked Read 4 Sent Done Done false
14422 4624 Acked Read 4 Sent Done Done false
14423 4625 Acked Read 3 Sent Done Done false
14424 4626 Acked Read 5 Sent Done Done false
14425 4627 Acked Read 1 Sent Done Done false
14426 4628 Acked Flush 0 Sent Done Done false
14427 STATES DS:0 DS:1 DS:2 TOTAL
14428 Sent 302 0 0 302
14429 Done 0 302 302 604
14430 Skipped 0 0 0 0
14431 Error 0 0 0 0
14432 Last Flush: 4326 4628 4628
14433 Downstairs last five completed: 4326 4325 4324 4323 4322
14434 Upstairs last five completed: 4628 4627 4626 4625 4624
14435 CLIENT: Up:0 ds:302 act:2
144362024-11-07T14:06:19.069ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144372024-11-07T14:06:19.069ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36660) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144382024-11-07T14:06:19.070ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144392024-11-07T14:06:19.070ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144402024-11-07T14:06:19.070ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144412024-11-07T14:06:19.070ZINFOcrucible: send last flush ID to this DS: 4326 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144422024-11-07T14:06:19.071ZINFOcrucible: Replied this last flush ID: 4326 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144432024-11-07T14:06:19.071ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144442024-11-07T14:06:19.071ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144452024-11-07T14:06:19.071ZINFOcrucible: [0] client re-new 302 jobs since flush 4326 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144462024-11-07T14:06:19.071ZINFOcrucible: [0] Replayed 302 jobs since flush: 4326 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
14447 ----------------------------------------------------------------
14448 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14449 Downstairs last five completed: 4628 4627 4626 4625 4624
14450 Upstairs last five completed: 4628 4627 4626 4625 4624
14451 CLIENT: Up:0 ds:0 act:3
14452 Replay: All jobs finished
14453 Main task has work to do, go find it
14454 got dsc Stop(1)
14455 stop 1
14456 Nov 07 14:06:21.364 INFO request completed, latency_us: 371, response_code: 204, uri: [8820] Got stop action so:false kr:false
14457 /stop/cid/1, method: POST, req_id: 2323b8f0-084f-4114-b991-1d99e0c2637f, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
144582024-11-07T14:06:21.365ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144592024-11-07T14:06:21.365ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144602024-11-07T14:06:21.365ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
14461 Nov 07 14:06:21.365 INFO request completed, latency_us: 161, response_code: 200, uri: /state/cid/1, method: GET, req_id: 81f290da-5533-42b8-9876-f246052cc017, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
144622024-11-07T14:06:21.365ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144632024-11-07T14:06:21.365ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
144642024-11-07T14:06:21.365ZINFOcrucible: client task is exiting = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
14465 [8820] Exited with: ExitStatus(unix_wait_status(9))
14466 [8820][1] reports Exit
14467 Nov 07 14:06:25.367 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3cf6404b-b3e7-470c-b975-c2f7fc4e77e2, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
14468 001/300 Read block 2446 len 8192
14469 002/300 Write block 250 len 16384 data: 3 3 2 2
14470 003/300 Write block 4366 len 24576 data: 4 5 5 5 5 5
14471 004/300 Read block 2191 len 4096
14472 005/300 Read block 3402 len 32768
14473 006/300 Write block 301 len 4096 data: 4
14474 007/300 Flush
14475 008/300 Write block 3985 len 8192 data: 3 3
14476 009/300 Read block 2886 len 8192
14477 010/300 Flush
14478 011/300 Write block 4061 len 8192 data: 2 2
14479 012/300 Write block 1206 len 4096 data: 3
14480 013/300 Flush
14481 014/300 Read block 1758 len 40960
14482 015/300 Read block 1572 len 4096
14483 016/300 Write block 3596 len 40960 data: 3 3 5 6 6 5 5 5 5 4
14484 017/300 Write block 4475 len 36864 data: 4 4 5 5 6 6 6 6 5
14485 018/300 Write block 4060 len 20480 data: 2 3 3 2 2
14486 019/300 Read block 2299 len 32768
14487 020/300 Write block 3140 len 28672 data: 3 3 4 3 2 2 2
14488 021/300 Read block 3595 len 32768
14489 022/300 Read block 1295 len 40960
14490 023/300 Flush
14491 024/300 Flush
14492 025/300 Flush
14493 026/300 Read block 4463 len 28672
14494 027/300 Read block 4703 len 36864
14495 028/300 Write block 4226 len 4096 data: 4
14496 029/300 Write block 714 len 8192 data: 3 2
14497 030/300 Read block 71 len 8192
14498 031/300 Read block 2775 len 16384
14499 032/300 Read block 4186 len 4096
14500 033/300 Read block 3616 len 24576
14501 034/300 Read block 570 len 40960
14502 035/300 Write block 794 len 8192 data: 6 6
14503 036/300 Write block 4237 len 24576 data: 3 4 5 4 4 3
14504 037/300 Read block 1640 len 4096
14505 038/300 Read block 32 len 16384
14506 039/300 Write block 2725 len 36864 data: 5 5 4 4 3 4 5 5 4
14507 040/300 Write block 4261 len 4096 data: 4
14508 041/300 Write block 1777 len 8192 data: 3 3
14509 042/300 Write block 2497 len 36864 data: 3 3 3 2 2 2 2 3 2
14510 043/300 Write block 3605 len 16384 data: 5 4 4 4
14511 044/300 Flush
14512 045/300 Read block 2905 len 40960
14513 046/300 Flush
14514 047/300 Write block 285 len 40960 data: 2 3 4 4 4 4 3 5 6 6
14515 048/300 Read block 3153 len 24576
14516 049/300 Write block 3076 len 40960 data: 4 3 3 2 3 5 5 5 5 4
14517 050/300 Read block 3856 len 40960
14518 051/300 Write block 2554 len 28672 data: 3 3 3 3 2 2 2
14519 052/300 Write block 2786 len 36864 data: 5 4 4 4 4 4 4 4 4
14520 053/300 Write block 3210 len 8192 data: 4 4
14521 054/300 Write block 2491 len 20480 data: 4 3 2 5 5
14522 055/300 Flush
14523 056/300 Write block 3507 len 32768 data: 4 4 5 5 5 4 3 3
14524 057/300 Read block 4436 len 40960
14525 058/300 Read block 152 len 4096
14526 059/300 Write block 2582 len 32768 data: 2 2 2 2 2 2 3 3
14527 060/300 Read block 1340 len 36864
14528 061/300 Write block 4464 len 8192 data: 4 4
14529 062/300 Read block 3833 len 36864
14530 063/300 Read block 4560 len 24576
14531 064/300 Write block 4669 len 16384 data: 3 2 2 4
14532 065/300 Flush
14533 066/300 Write block 4962 len 12288 data: 3 3 3
14534 067/300 Read block 1254 len 8192
14535 068/300 Flush
14536 069/300 Write block 1830 len 40960 data: 4 4 4 3 2 2 3 4 3 3
14537 070/300 Write block 2450 len 40960 data: 2 2 2 2 3 3 3 3 3 3
14538 071/300 Write block 4554 len 24576 data: 4 4 5 5 6 7
14539 072/300 Flush
14540 073/300 Read block 659 len 16384
14541 074/300 Read block 1151 len 12288
14542 075/300 Write block 2816 len 24576 data: 3 4 4 4 4 3
14543 076/300 Write block 444 len 12288 data: 5 4 3
14544 077/300 Read block 4580 len 12288
14545 078/300 Write block 178 len 8192 data: 2 3
14546 079/300 Read block 127 len 8192
14547 080/300 Read block 1207 len 32768
14548 081/300 Read block 2762 len 32768
14549 082/300 Write block 476 len 40960 data: 3 4 3 3 4 4 5 4 4 5
14550 083/300 Flush
14551 084/300 Read block 2436 len 16384
14552 085/300 Read block 3054 len 20480
14553 086/300 Write block 3463 len 12288 data: 4 4 3
14554 087/300 Flush
14555 088/300 Write block 1198 len 20480 data: 3 3 3 3 3
14556 089/300 Flush
14557 090/300 Flush
14558 091/300 Read block 4021 len 40960
14559 092/300 Read block 371 len 16384
14560 093/300 Read block 2088 len 16384
14561 094/300 Write block 324 len 40960 data: 3 4 3 3 3 3 3 2 2 3
14562 095/300 Read block 3452 len 12288
14563 096/300 Write block 2801 len 12288 data: 3 3 3
14564 097/300 Read block 716 len 8192
14565 098/300 Read block 3373 len 36864
14566 099/300 Write block 4097 len 40960 data: 2 2 3 4 4 4 4 4 5 5
14567 100/300 Read block 3502 len 36864
14568 101/300 Read block 4541 len 16384
14569 102/300 Write block 4164 len 8192 data: 3 2
14570 103/300 Write block 1562 len 4096 data: 4
14571 104/300 Write block 4726 len 16384 data: 3 4 4 4
14572 105/300 Read block 351 len 12288
14573 106/300 Write block 4841 len 40960 data: 3 5 5 5 4 4 4 2 2 2
14574 107/300 Read block 4325 len 12288
14575 108/300 Read block 3328 len 40960
14576 109/300 Read block 3425 len 40960
14577 110/300 Read block 2011 len 28672
14578 111/300 Flush
14579 112/300 Read block 4617 len 36864
14580 113/300 Read block 322 len 4096
14581 114/300 Read block 2628 len 32768
14582 115/300 Write block 3674 len 16384 data: 2 2 2 2
14583 116/300 Write block 4007 len 24576 data: 4 3 5 6 6 7
14584 117/300 Read block 3207 len 24576
14585 118/300 Read block 1771 len 4096
14586 119/300 Flush
14587 120/300 Flush
14588 121/300 Write block 2915 len 24576 data: 3 4 3 3 3 3
14589 122/300 Read block 3191 len 4096
14590 123/300 Read block 619 len 8192
14591 124/300 Write block 4263 len 40960 data: 4 4 4 4 2 4 5 5 5 5
14592 125/300 Write block 4788 len 32768 data: 5 5 5 4 4 2 3 3
14593 126/300 Write block 969 len 40960 data: 2 2 3 2 2 2 2 3 3 3
14594 127/300 Read block 3943 len 32768
14595 128/300 Read block 4508 len 16384
14596 129/300 Read block 3974 len 16384
14597 130/300 Write block 2591 len 4096 data: 4
14598 131/300 Read block 273 len 40960
14599 132/300 Read block 2751 len 4096
14600 133/300 Write block 531 len 20480 data: 4 4 4 3 3
14601 134/300 Write block 12 len 8192 data: 4 5
14602 135/300 Write block 1046 len 8192 data: 5 5
14603 136/300 Write block 1778 len 8192 data: 4 4
14604 137/300 Read block 1853 len 36864
14605 138/300 Write block 2471 len 16384 data: 3 4 4 4
14606 139/300 Write block 2616 len 24576 data: 3 4 4 3 4 3
14607 140/300 Read block 2901 len 16384
14608 141/300 Write block 4485 len 16384 data: 5 4 4 3
14609 142/300 Read block 4722 len 40960
14610 143/300 Write block 1199 len 28672 data: 4 4 4 4 2 2 2
14611 144/300 Write block 2535 len 36864 data: 3 3 3 3 7 7 7 6 6
14612 145/300 Write block 3555 len 8192 data: 6 5
14613 146/300 Read block 2316 len 4096
14614 147/300 Read block 501 len 28672
14615 148/300 Flush
14616 149/300 Read block 665 len 16384
14617 150/300 Write block 2538 len 32768 data: 4 8 8 8 7 7 6 5
14618 151/300 Read block 1033 len 24576
14619 152/300 Flush
14620 153/300 Read block 4324 len 32768
14621 154/300 Write block 884 len 32768 data: 4 3 2 3 3 3 3 4
14622 155/300 Read block 2036 len 12288
14623 156/300 Read block 1922 len 20480
14624 157/300 Write block 1084 len 4096 data: 4
14625 158/300 Flush
14626 159/300 Write block 4696 len 8192 data: 4 6
14627 160/300 Write block 3439 len 4096 data: 4
14628 161/300 Read block 3392 len 4096
14629 162/300 Write block 2631 len 20480 data: 3 3 3 2 2
14630 163/300 Read block 2095 len 36864
14631 164/300 Write block 4954 len 40960 data: 6 6 5 6 5 5 5 4 4 4
14632 165/300 Write block 2827 len 20480 data: 6 5 5 4 4
14633 166/300 Read block 1910 len 32768
14634 167/300 Write block 828 len 28672 data: 3 3 3 3 3 3 3
14635 168/300 Read block 1527 len 28672
14636 169/300 Read block 3390 len 20480
14637 170/300 Write block 3956 len 40960 data: 3 2 2 2 2 2 2 3 3 3
14638 171/300 Read block 3946 len 8192
14639 172/300 Write block 2165 len 20480 data: 5 5 4 4 4
14640 173/300 Read block 533 len 36864
14641 174/300 Read block 4388 len 12288
14642 175/300 Read block 2146 len 12288
14643 176/300 Write block 2940 len 40960 data: 2 2 3 3 3 3 3 3 3 2
14644 177/300 Write block 1502 len 32768 data: 2 2 2 3 3 3 3 4
14645 178/300 Read block 640 len 32768
14646 179/300 Write block 256 len 4096 data: 4
14647 180/300 Flush
14648 181/300 Write block 706 len 8192 data: 6 7
14649 182/300 Read block 3079 len 12288
14650 183/300 Flush
14651 184/300 Write block 2426 len 16384 data: 5 5 5 4
14652 185/300 Write block 1301 len 12288 data: 3 3 3
14653 186/300 Read block 1015 len 24576
14654 187/300 Flush
14655 188/300 Write block 1493 len 28672 data: 2 2 2 2 2 2 2
14656 189/300 Flush
14657 190/300 Write block 871 len 12288 data: 4 5 5
14658 191/300 Write block 1135 len 12288 data: 4 4 4
14659 192/300 Read block 3307 len 40960
14660 193/300 Write block 4311 len 40960 data: 3 3 4 6 5 5 5 5 5 4
14661 194/300 Write block 4271 len 20480 data: 6 6 5 4 4
14662 195/300 Write block 2976 len 36864 data: 4 4 4 3 3 2 2 2 2
14663 196/300 Read block 3664 len 32768
14664 197/300 Read block 3939 len 8192
14665 198/300 Flush
14666 199/300 Write block 1419 len 12288 data: 3 3 3
14667 200/300 Read block 67 len 40960
14668 201/300 Read block 3956 len 24576
14669 202/300 Read block 3558 len 40960
14670 203/300 Flush
14671 204/300 Write block 4520 len 16384 data: 3 4 4 4
14672 205/300 Flush
14673 206/300 Write block 2833 len 16384 data: 3 3 3 3
14674 207/300 Read block 1226 len 16384
14675 208/300 Write block 1277 len 40960 data: 2 3 3 3 3 3 3 3 3 3
14676 209/300 Read block 3100 len 28672
14677 210/300 Write block 674 len 4096 data: 3
14678 211/300 Read block 3358 len 24576
14679 212/300 Flush
14680 213/300 Write block 2448 len 24576 data: 2 2 3 3 3 3
14681 214/300 Read block 2495 len 36864
14682 215/300 Write block 4309 len 12288 data: 2 2 4
14683 216/300 Read block 2346 len 28672
14684 217/300 Write block 1124 len 16384 data: 4 4 4 3
14685 218/300 Write block 2164 len 32768 data: 5 6 6 5 5 5 4 3
14686 219/300 Read block 2505 len 32768
14687 220/300 Flush
14688 221/300 Write block 3751 len 12288 data: 5 5 6
14689 222/300 Read block 4280 len 32768
14690 223/300 Write block 3590 len 24576 data: 4 4 2 2 2 3
14691 224/300 Read block 2302 len 40960
14692 225/300 Read block 1371 len 40960
14693 226/300 Flush
14694 227/300 Read block 2841 len 40960
14695 228/300 Write block 4144 len 36864 data: 3 3 4 4 6 6 7 6 5
14696 229/300 Flush
14697 230/300 Read block 3188 len 28672
14698 231/300 Write block 4043 len 32768 data: 4 5 5 4 4 4 3 3
14699 232/300 Read block 2235 len 36864
14700 233/300 Read block 3832 len 24576
14701 234/300 Flush
14702 235/300 Write block 3376 len 40960 data: 6 6 5 4 4 4 3 3 3 3
14703 236/300 Read block 2115 len 12288
14704 237/300 Write block 4436 len 28672 data: 6 5 6 7 5 5 5
14705 238/300 Flush
14706 239/300 Read block 215 len 8192
14707 240/300 Read block 3444 len 28672
14708 241/300 Write block 4596 len 8192 data: 3 4
14709 242/300 Write block 2040 len 24576 data: 2 2 2 3 3 3
14710 243/300 Write block 3938 len 4096 data: 3
14711 244/300 Write block 2277 len 28672 data: 7 5 5 4 2 2 2
14712 245/300 Write block 2909 len 28672 data: 3 4 5 5 4 4 4
14713 246/300 Write block 4996 len 4096 data: 2
14714 247/300 Write block 2985 len 20480 data: 2 2 2 2 2
14715 248/300 Write block 697 len 4096 data: 4
14716 249/300 Read block 2248 len 4096
14717 250/300 Write block 126 len 4096 data: 3
14718 251/300 Write block 1098 len 36864 data: 3 3 4 4 3 3 4 4 4
14719 252/300 Read block 4147 len 4096
14720 253/300 Write block 1538 len 12288 data: 2 3 3
14721 254/300 Read block 3212 len 12288
14722 255/300 Write block 4524 len 20480 data: 4 4 6 6 6
14723 256/300 Write block 2491 len 40960 data: 5 4 3 6 6 5 4 4 4 3
14724 257/300 Read block 291 len 28672
14725 258/300 Write block 4115 len 32768 data: 4 3 3 3 5 4 4 4
14726 259/300 Flush
14727 260/300 Read block 1947 len 8192
14728 261/300 Read block 474 len 40960
14729 262/300 Read block 930 len 8192
14730 263/300 Read block 1467 len 36864
14731 264/300 Write block 4915 len 24576 data: 4 4 4 4 5 5
14732 265/300 Read block 864 len 12288
14733 266/300 Read block 3313 len 4096
14734 267/300 Write block 4727 len 24576 data: 5 5 5 4 4 3
14735 268/300 Read block 1359 len 20480
14736 269/300 Write block 4137 len 28672 data: 4 4 4 4 4 2 3
14737 270/300 Read block 4951 len 28672
14738 271/300 Flush
14739 272/300 Write block 3860 len 4096 data: 3
14740 273/300 Read block 4463 len 8192
14741 274/300 Read block 2990 len 20480
14742 275/300 Write block 3686 len 4096 data: 5
14743 276/300 Read block 468 len 36864
14744 277/300 Read block 4094 len 24576
14745 278/300 Flush
14746 279/300 Write block 2471 len 4096 data: 4
14747 280/300 Write block 3387 len 20480 data: 4 3 2 2 2
14748 281/300 Flush
14749 282/300 Write block 4223 len 40960 data: 4 4 5 5 5 5 5 4 3 3
14750 283/300 Write block 1222 len 20480 data: 4 3 3 3 3
14751 284/300 Write block 503 len 20480 data: 3 3 2 2 2
14752 285/300 Read block 4926 len 32768
14753 286/300 Write block 1887 len 24576 data: 4 4 4 4 4 2
14754 287/300 Write block 3918 len 28672 data: 2 2 2 3 3 4 6
14755 288/300 Write block 2200 len 32768 data: 2 3 3 2 2 3 4 4
14756 289/300 Read block 943 len 36864
14757 290/300 Write block 3267 len 4096 data: 5
14758 291/300 Flush
14759 292/300 Read block 425 len 8192
14760 293/300 Read block 1919 len 8192
14761 294/300 Flush
14762 295/300 Read block 2752 len 8192
14763 296/300 Read block 4886 len 16384
14764 297/300 Read block 901 len 40960
14765 298/300 Write block 760 len 40960 data: 3 3 5 4 3 3 2 2 3 3
14766 299/300 Write block 4041 len 8192 data: 4 4
14767 300/300 Read block 3144 len 28672
14768 301/300 Write block 3550 len 16384 data: 3 4 6 6
14769 Main task has work to do, go find it
14770 got dsc Start(1)
14771 start 1
14772 Nov 07 14:06:25.659 INFO request completed, latency_us: 235, response_code: 204, uri: /start/cid/1, method: POST, req_id: c218a00c-89ab-4c99-8f82-93d3bd53bc83, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
14773 Replay: started 1, returned:Ok(())
14774 ----------------------------------------------------------------
14775 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14776 States: Active Offline Active
14777 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14778 4629 Acked Read 2 Done Sent Done false
14779 4630 Acked Write 4 Done Sent Done false
14780 4631 Acked Write 6 Done Sent Done false
14781 4632 Acked Read 1 Done Sent Done false
14782 4633 Acked Read 8 Done Sent Done false
14783 4634 Acked Write 1 Done Sent Done false
14784 4635 Acked Flush 0 Done Sent Done false
14785 4636 Acked Write 2 Done Sent Done false
14786 4637 Acked Read 2 Done Sent Done false
14787 4638 Acked Flush 0 Done Sent Done false
14788 4639 Acked Write 2 Done Sent Done false
14789 4640 Acked Write 1 Done Sent Done false
14790 4641 Acked Flush 0 Done Sent Done false
14791 4642 Acked Read 10 Done Sent Done false
14792 4643 Acked Read 1 Done Sent Done false
14793 4644 Acked Write 10 Done Sent Done false
14794 4645 Acked Write 9 Done Sent Done false
14795 4646 Acked Write 5 Done Sent Done false
14796 4647 Acked Read 8 Done Sent Done false
14797 4648 Acked Write 7 Done Sent Done false
14798 4649 Acked Read 8 Done Sent Done false
14799 4650 Acked Read 10 Done Sent Done false
14800 4651 Acked Flush 0 Done Sent Done false
14801 4652 Acked Flush 0 Done Sent Done false
14802 4653 Acked Flush 0 Done Sent Done false
14803 4654 Acked Read 7 Done Sent Done false
14804 4655 Acked Read 9 Done Sent Done false
14805 4656 Acked Write 1 Done Sent Done false
14806 4657 Acked Write 2 Done Sent Done false
14807 4658 Acked Read 2 Done Sent Done false
14808 4659 Acked Read 4 Done Sent Done false
14809 4660 Acked Read 1 Done Sent Done false
14810 4661 Acked Read 6 Done Sent Done false
14811 4662 Acked Read 10 Done Sent Done false
14812 4663 Acked Write 2 Done Sent Done false
14813 4664 Acked Write 6 Done Sent Done false
14814 4665 Acked Read 1 Done Sent Done false
14815 4666 Acked Read 4 Done Sent Done false
14816 4667 Acked Write 9 Done Sent Done false
14817 4668 Acked Write 1 Done Sent Done false
14818 4669 Acked Write 2 Done Sent Done false
14819 4670 Acked Write 9 Done Sent Done false
14820 4671 Acked Write 4 Done Sent Done false
14821 4672 Acked Flush 0 Done Sent Done false
14822 4673 Acked Read 10 Done Sent Done false
14823 4674 Acked Flush 0 Done Sent Done false
14824 4675 Acked Write 10 Done Sent Done false
14825 4676 Acked Read 6 Done Sent Done false
14826 4677 Acked Write 10 Done Sent Done false
14827 4678 Acked Read 10 Done Sent Done false
14828 4679 Acked Write 7 Done Sent Done false
14829 4680 Acked Write 9 Done Sent Done false
14830 4681 Acked Write 2 Done Sent Done false
14831 4682 Acked Write 5 Done Sent Done false
14832 4683 Acked Flush 0 Done Sent Done false
14833 4684 Acked Write 8 Done Sent Done false
14834 4685 Acked Read 10 Done Sent Done false
14835 4686 Acked Read 1 Done Sent Done false
14836 4687 Acked Write 8 Done Sent Done false
14837 4688 Acked Read 9 Done Sent Done false
14838 4689 Acked Write 2 Done Sent Done false
14839 4690 Acked Read 9 Done Sent Done false
14840 4691 Acked Read 6 Done Sent Done false
14841 4692 Acked Write 4 Done Sent Done false
14842 4693 Acked Flush 0 Done Sent Done false
14843 4694 Acked Write 3 Done Sent Done false
14844 4695 Acked Read 2 Done Sent Done false
14845 4696 Acked Flush 0 Done Sent Done false
14846 4697 Acked Write 10 Done Sent Done false
14847 4698 Acked Write 10 Done Sent Done false
14848 4699 Acked Write 6 Done Sent Done false
14849 4700 Acked Flush 0 Done Sent Done false
14850 4701 Acked Read 4 Done Sent Done false
14851 4702 Acked Read 3 Done Sent Done false
14852 4703 Acked Write 6 Done Sent Done false
14853 4704 Acked Write 3 Done Sent Done false
14854 4705 Acked Read 3 Done Sent Done false
14855 4706 Acked Write 2 Done Sent Done false
14856 4707 Acked Read 2 Done Sent Done false
14857 4708 Acked Read 8 Done Sent Done false
14858 4709 Acked Read 8 Done Sent Done false
14859 4710 Acked Write 10 Done Sent Done false
14860 4711 Acked Flush 0 Done Sent Done false
14861 4712 Acked Read 4 Done Sent Done false
14862 4713 Acked Read 5 Done Sent Done false
14863 4714 Acked Write 3 Done Sent Done false
14864 4715 Acked Flush 0 Done Sent Done false
14865 4716 Acked Write 5 Done Sent Done false
14866 4717 Acked Flush 0 Done Sent Done false
14867 4718 Acked Flush 0 Done Sent Done false
14868 4719 Acked Read 10 Done Sent Done false
14869 4720 Acked Read 4 Done Sent Done false
14870 4721 Acked Read 4 Done Sent Done false
14871 4722 Acked Write 10 Done Sent Done false
14872 4723 Acked Read 3 Done Sent Done false
14873 4724 Acked Write 3 Done Sent Done false
14874 4725 Acked Read 2 Done Sent Done false
14875 4726 Acked Read 9 Done Sent Done false
14876 4727 Acked Write 10 Done Sent Done false
14877 4728 Acked Read 9 Done Sent Done false
14878 4729 Acked Read 4 Done Sent Done false
14879 4730 Acked Write 2 Done Sent Done false
14880 4731 Acked Write 1 Done Sent Done false
14881 4732 Acked Write 4 Done Sent Done false
14882 4733 Acked Read 3 Done Sent Done false
14883 4734 Acked Write 10 Done Sent Done false
14884 4735 Acked Read 3 Done Sent Done false
14885 4736 Acked Read 10 Done Sent Done false
14886 4737 Acked Read 10 Done Sent Done false
14887 4738 Acked Read 7 Done Sent Done false
14888 4739 Acked Flush 0 Done Sent Done false
14889 4740 Acked Read 9 Done Sent Done false
14890 4741 Acked Read 1 Done Sent Done false
14891 4742 Acked Read 8 Done Sent Done false
14892 4743 Acked Write 4 Done Sent Done false
14893 4744 Acked Write 6 Done Sent Done false
14894 4745 Acked Read 6 Done Sent Done false
14895 4746 Acked Read 1 Done Sent Done false
14896 4747 Acked Flush 0 Done Sent Done false
14897 4748 Acked Flush 0 Done Sent Done false
14898 4749 Acked Write 6 Done Sent Done false
14899 4750 Acked Read 1 Done Sent Done false
14900 4751 Acked Read 2 Done Sent Done false
14901 4752 Acked Write 10 Done Sent Done false
14902 4753 Acked Write 8 Done Sent Done false
14903 4754 Acked Write 10 Done Sent Done false
14904 4755 Acked Read 8 Done Sent Done false
14905 4756 Acked Read 4 Done Sent Done false
14906 4757 Acked Read 4 Done Sent Done false
14907 4758 Acked Write 1 Done Sent Done false
14908 4759 Acked Read 10 Done Sent Done false
14909 4760 Acked Read 1 Done Sent Done false
14910 4761 Acked Write 5 Done Sent Done false
14911 4762 Acked Write 2 Done Sent Done false
14912 4763 Acked Write 2 Done Sent Done false
14913 4764 Acked Write 2 Done Sent Done false
14914 4765 Acked Read 9 Done Sent Done false
14915 4766 Acked Write 4 Done Sent Done false
14916 4767 Acked Write 6 Done Sent Done false
14917 4768 Acked Read 4 Done Sent Done false
14918 4769 Acked Write 4 Done Sent Done false
14919 4770 Acked Read 10 Done Sent Done false
14920 4771 Acked Write 7 Done Sent Done false
14921 4772 Acked Write 9 Done Sent Done false
14922 4773 Acked Write 2 Done Sent Done false
14923 4774 Acked Read 1 Done Sent Done false
14924 4775 Acked Read 7 Done Sent Done false
14925 4776 Acked Flush 0 Done Sent Done false
14926 4777 Acked Read 4 Done Sent Done false
14927 4778 Acked Write 8 Done Sent Done false
14928 4779 Acked Read 6 Done Sent Done false
14929 4780 Acked Flush 0 Done Sent Done false
14930 4781 Acked Read 8 Done Sent Done false
14931 4782 Acked Write 8 Done Sent Done false
14932 4783 Acked Read 3 Done Sent Done false
14933 4784 Acked Read 5 Done Sent Done false
14934 4785 Acked Write 1 Done Sent Done false
14935 4786 Acked Flush 0 Done Sent Done false
14936 4787 Acked Write 2 Done Sent Done false
14937 4788 Acked Write 1 Done Sent Done false
14938 4789 Acked Read 1 Done Sent Done false
14939 4790 Acked Write 5 Done Sent Done false
14940 4791 Acked Read 9 Done Sent Done false
14941 4792 Acked Write 10 Done Sent Done false
14942 4793 Acked Write 5 Done Sent Done false
14943 4794 Acked Read 8 Done Sent Done false
14944 4795 Acked Write 7 Done Sent Done false
14945 4796 Acked Read 7 Done Sent Done false
14946 4797 Acked Read 5 Done Sent Done false
14947 4798 Acked Write 10 Done Sent Done false
14948 4799 Acked Read 2 Done Sent Done false
14949 4800 Acked Write 5 Done Sent Done false
14950 4801 Acked Read 9 Done Sent Done false
14951 4802 Acked Read 3 Done Sent Done false
14952 4803 Acked Read 3 Done Sent Done false
14953 4804 Acked Write 10 Done Sent Done false
14954 4805 Acked Write 8 Done Sent Done false
14955 4806 Acked Read 8 Done Sent Done false
14956 4807 Acked Write 1 Done Sent Done false
14957 4808 Acked Flush 0 Done Sent Done false
14958 4809 Acked Write 2 Done Sent Done false
14959 4810 Acked Read 3 Done Sent Done false
14960 4811 Acked Flush 0 Done Sent Done false
14961 4812 Acked Write 4 Done Sent Done false
14962 4813 Acked Write 3 Done Sent Done false
14963 4814 Acked Read 6 Done Sent Done false
14964 4815 Acked Flush 0 Done Sent Done false
14965 4816 Acked Write 7 Done Sent Done false
14966 4817 Acked Flush 0 Done Sent Done false
14967 4818 Acked Write 3 Done Sent Done false
14968 4819 Acked Write 3 Done Sent Done false
14969 4820 Acked Read 10 Done Sent Done false
14970 4821 Acked Write 10 Done Sent Done false
14971 4822 Acked Write 5 Done Sent Done false
14972 4823 Acked Write 9 Done Sent Done false
14973 4824 Acked Read 8 Done Sent Done false
14974 4825 Acked Read 2 Done Sent Done false
14975 4826 Acked Flush 0 Done Sent Done false
14976 4827 Acked Write 3 Done Sent Done false
14977 4828 Acked Read 10 Done Sent Done false
14978 4829 Acked Read 6 Done Sent Done false
14979 4830 Acked Read 10 Done Sent Done false
14980 4831 Acked Flush 0 Done Sent Done false
14981 4832 Acked Write 4 Done Sent Done false
14982 4833 Acked Flush 0 Done Sent Done false
14983 4834 Acked Write 4 Done Sent Done false
14984 4835 Acked Read 4 Done Sent Done false
14985 4836 Acked Write 10 Done Sent Done false
14986 4837 Acked Read 7 Done Sent Done false
14987 4838 Acked Write 1 Done Sent Done false
14988 4839 Acked Read 6 Done Sent Done false
14989 4840 Acked Flush 0 Done Sent Done false
14990 4841 Acked Write 6 Done Sent Done false
14991 4842 Acked Read 9 Done Sent Done false
14992 4843 Acked Write 3 Done Sent Done false
14993 4844 Acked Read 7 Done Sent Done false
14994 4845 Acked Write 4 Done Sent Done false
14995 4846 Acked Write 8 Done Sent Done false
14996 4847 Acked Read 8 Done Sent Done false
14997 4848 Acked Flush 0 Done Sent Done false
14998 4849 Acked Write 3 Done Sent Done false
14999 4850 Acked Read 8 Done Sent Done false
15000 4851 Acked Write 6 Done Sent Done false
15001 4852 Acked Read 10 Done Sent Done false
15002 4853 Acked Read 10 Done Sent Done false
15003 4854 Acked Flush 0 Done Sent Done false
15004 4855 Acked Read 10 Done Sent Done false
15005 4856 Acked Write 9 Done Sent Done false
15006 4857 Acked Flush 0 Done Sent Done false
15007 4858 Acked Read 7 Done Sent Done false
15008 4859 Acked Write 8 Done Sent Done false
15009 4860 Acked Read 9 Done Sent Done false
15010 4861 Acked Read 6 Done Sent Done false
15011 4862 Acked Flush 0 Done Sent Done false
15012 4863 Acked Write 10 Done Sent Done false
15013 4864 Acked Read 3 Done Sent Done false
15014 4865 Acked Write 7 Done Sent Done false
15015 4866 Acked Flush 0 Done Sent Done false
15016 4867 Acked Read 2 Done Sent Done false
15017 4868 Acked Read 7 Done Sent Done false
15018 4869 Acked Write 2 Done Sent Done false
15019 4870 Acked Write 6 Done Sent Done false
15020 4871 Acked Write 1 Done Sent Done false
15021 4872 Acked Write 7 Done Sent Done false
15022 4873 Acked Write 7 Done Sent Done false
15023 4874 Acked Write 1 Done Sent Done false
15024 4875 Acked Write 5 Done Sent Done false
15025 4876 Acked Write 1 Done Sent Done false
15026 4877 Acked Read 1 Done Sent Done false
15027 4878 Acked Write 1 Done Sent Done false
15028 4879 Acked Write 9 Done Sent Done false
15029 4880 Acked Read 1 Done Sent Done false
15030 4881 Acked Write 3 Done Sent Done false
15031 4882 Acked Read 3 Done Sent Done false
15032 4883 Acked Write 5 Done Sent Done false
15033 4884 Acked Write 10 Done Sent Done false
15034 4885 Acked Read 7 Done Sent Done false
15035 4886 Acked Flush 0 Done Sent Done false
15036 4887 Acked Write 8 Done Sent Done false
15037 4888 Acked Flush 0 Done Sent Done false
15038 4889 Acked Read 2 Done Sent Done false
15039 4890 Acked Read 10 Done Sent Done false
15040 4891 Acked Read 2 Done Sent Done false
15041 4892 Acked Read 9 Done Sent Done false
15042 4893 Acked Write 6 Done Sent Done false
15043 4894 Acked Read 3 Done Sent Done false
15044 4895 Acked Read 1 Done Sent Done false
15045 4896 Acked Write 6 Done Sent Done false
15046 4897 Acked Read 5 Done Sent Done false
15047 4898 Acked Write 7 Done Sent Done false
15048 4899 Acked Read 7 Done Sent Done false
15049 4900 Acked Flush 0 Done Sent Done false
15050 4901 Acked Write 1 Done Sent Done false
15051 4902 Acked Read 2 Done Sent Done false
15052 4903 Acked Read 5 Done Sent Done false
15053 4904 Acked Write 1 Done Sent Done false
15054 4905 Acked Read 9 Done Sent Done false
15055 4906 Acked Read 6 Done Sent Done false
15056 4907 Acked Flush 0 Done Sent Done false
15057 4908 Acked Write 1 Done Sent Done false
15058 4909 Acked Write 5 Done Sent Done false
15059 4910 Acked Flush 0 Done Sent Done false
15060 4911 Acked Write 10 Done Sent Done false
15061 4912 Acked Write 5 Done Sent Done false
15062 4913 Acked Write 5 Done Sent Done false
15063 4914 Acked Read 8 Done Sent Done false
15064 4915 Acked Write 6 Done Sent Done false
15065 4916 Acked Write 7 Done Sent Done false
15066 4917 Acked Write 8 Done Sent Done false
15067 4918 Acked Read 9 Done Sent Done false
15068 4919 Acked Write 1 Done Sent Done false
15069 4920 Acked Flush 0 Done Sent Done false
15070 4921 Acked Read 2 Done Sent Done false
15071 4922 Acked Read 2 Done Sent Done false
15072 4923 Acked Flush 0 Done Sent Done false
15073 4924 Acked Read 2 Done Sent Done false
15074 4925 Acked Read 4 Done Sent Done false
15075 4926 Acked Read 10 Done Sent Done false
15076 4927 Acked Write 10 Done Sent Done false
15077 4928 Acked Write 2 Done Sent Done false
15078 4929 Acked Read 7 Done Sent Done false
15079 4930 Acked Write 4 Done Sent Done false
15080 STATES DS:0 DS:1 DS:2 TOTAL
15081 Sent 0 302 0 302
15082 Done 302 0 302 604
15083 Skipped 0 0 0 0
15084 Error 0 0 0 0
15085 Last Flush: 4923 4628 4923
15086 Downstairs last five completed: 4628 4627 4626 4625 4624
15087 Upstairs last five completed: 4930 4929 4928 4927 4926
15088 CLIENT: Up:0 ds:302 act:2
15089 [8820] got start action
15090 [8820] I am going to restart
15091 Starting downstairs at port 8820
15092 Make output file at "/tmp/dsc/downstairs-8820.txt"
15093 [8820][1] reports Starting
15094 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1212)
15095 [8820][1] reports Running
15096 ----------------------------------------------------------------
15097 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
15098 States: Active Offline Active
15099 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15100 4629 Acked Read 2 Done Sent Done false
15101 4630 Acked Write 4 Done Sent Done false
15102 4631 Acked Write 6 Done Sent Done false
15103 4632 Acked Read 1 Done Sent Done false
15104 4633 Acked Read 8 Done Sent Done false
15105 4634 Acked Write 1 Done Sent Done false
15106 4635 Acked Flush 0 Done Sent Done false
15107 4636 Acked Write 2 Done Sent Done false
15108 4637 Acked Read 2 Done Sent Done false
15109 4638 Acked Flush 0 Done Sent Done false
15110 4639 Acked Write 2 Done Sent Done false
15111 4640 Acked Write 1 Done Sent Done false
15112 4641 Acked Flush 0 Done Sent Done false
15113 4642 Acked Read 10 Done Sent Done false
15114 4643 Acked Read 1 Done Sent Done false
15115 4644 Acked Write 10 Done Sent Done false
15116 4645 Acked Write 9 Done Sent Done false
15117 4646 Acked Write 5 Done Sent Done false
15118 4647 Acked Read 8 Done Sent Done false
15119 4648 Acked Write 7 Done Sent Done false
15120 4649 Acked Read 8 Done Sent Done false
15121 4650 Acked Read 10 Done Sent Done false
15122 4651 Acked Flush 0 Done Sent Done false
15123 4652 Acked Flush 0 Done Sent Done false
15124 4653 Acked Flush 0 Done Sent Done false
15125 4654 Acked Read 7 Done Sent Done false
15126 4655 Acked Read 9 Done Sent Done false
15127 4656 Acked Write 1 Done Sent Done false
15128 4657 Acked Write 2 Done Sent Done false
15129 4658 Acked Read 2 Done Sent Done false
15130 4659 Acked Read 4 Done Sent Done false
15131 4660 Acked Read 1 Done Sent Done false
15132 4661 Acked Read 6 Done Sent Done false
15133 4662 Acked Read 10 Done Sent Done false
15134 4663 Acked Write 2 Done Sent Done false
15135 4664 Acked Write 6 Done Sent Done false
15136 4665 Acked Read 1 Done Sent Done false
15137 4666 Acked Read 4 Done Sent Done false
15138 4667 Acked Write 9 Done Sent Done false
15139 4668 Acked Write 1 Done Sent Done false
15140 4669 Acked Write 2 Done Sent Done false
15141 4670 Acked Write 9 Done Sent Done false
15142 4671 Acked Write 4 Done Sent Done false
15143 4672 Acked Flush 0 Done Sent Done false
15144 4673 Acked Read 10 Done Sent Done false
15145 4674 Acked Flush 0 Done Sent Done false
15146 4675 Acked Write 10 Done Sent Done false
15147 4676 Acked Read 6 Done Sent Done false
15148 4677 Acked Write 10 Done Sent Done false
15149 4678 Acked Read 10 Done Sent Done false
15150 4679 Acked Write 7 Done Sent Done false
15151 4680 Acked Write 9 Done Sent Done false
15152 4681 Acked Write 2 Done Sent Done false
15153 4682 Acked Write 5 Done Sent Done false
15154 4683 Acked Flush 0 Done Sent Done false
15155 4684 Acked Write 8 Done Sent Done false
15156 4685 Acked Read 10 Done Sent Done false
15157 4686 Acked Read 1 Done Sent Done false
15158 4687 Acked Write 8 Done Sent Done false
15159 4688 Acked Read 9 Done Sent Done false
15160 4689 Acked Write 2 Done Sent Done false
15161 4690 Acked Read 9 Done Sent Done false
15162 4691 Acked Read 6 Done Sent Done false
15163 4692 Acked Write 4 Done Sent Done false
15164 4693 Acked Flush 0 Done Sent Done false
15165 4694 Acked Write 3 Done Sent Done false
15166 4695 Acked Read 2 Done Sent Done false
15167 4696 Acked Flush 0 Done Sent Done false
15168 4697 Acked Write 10 Done Sent Done false
15169 4698 Acked Write 10 Done Sent Done false
15170 4699 Acked Write 6 Done Sent Done false
15171 4700 Acked Flush 0 Done Sent Done false
15172 4701 Acked Read 4 Done Sent Done false
15173 4702 Acked Read 3 Done Sent Done false
15174 4703 Acked Write 6 Done Sent Done false
15175 4704 Acked Write 3 Done Sent Done false
15176 4705 Acked Read 3 Done Sent Done false
15177 4706 Acked Write 2 Done Sent Done false
15178 4707 Acked Read 2 Done Sent Done false
15179 4708 Acked Read 8 Done Sent Done false
15180 4709 Acked Read 8 Done Sent Done false
15181 4710 Acked Write 10 Done Sent Done false
15182 4711 Acked Flush 0 Done Sent Done false
15183 4712 Acked Read 4 Done Sent Done false
15184 4713 Acked Read 5 Done Sent Done false
15185 4714 Acked Write 3 Done Sent Done false
15186 4715 Acked Flush 0 Done Sent Done false
15187 4716 Acked Write 5 Done Sent Done false
15188 4717 Acked Flush 0 Done Sent Done false
15189 4718 Acked Flush 0 Done Sent Done false
15190 4719 Acked Read 10 Done Sent Done false
15191 4720 Acked Read 4 Done Sent Done false
15192 4721 Acked Read 4 Done Sent Done false
15193 4722 Acked Write 10 Done Sent Done false
15194 4723 Acked Read 3 Done Sent Done false
15195 4724 Acked Write 3 Done Sent Done false
15196 4725 Acked Read 2 Done Sent Done false
15197 4726 Acked Read 9 Done Sent Done false
15198 4727 Acked Write 10 Done Sent Done false
15199 4728 Acked Read 9 Done Sent Done false
15200 4729 Acked Read 4 Done Sent Done false
15201 4730 Acked Write 2 Done Sent Done false
15202 4731 Acked Write 1 Done Sent Done false
15203 4732 Acked Write 4 Done Sent Done false
15204 4733 Acked Read 3 Done Sent Done false
15205 4734 Acked Write 10 Done Sent Done false
15206 4735 Acked Read 3 Done Sent Done false
15207 4736 Acked Read 10 Done Sent Done false
15208 4737 Acked Read 10 Done Sent Done false
15209 4738 Acked Read 7 Done Sent Done false
15210 4739 Acked Flush 0 Done Sent Done false
15211 4740 Acked Read 9 Done Sent Done false
15212 4741 Acked Read 1 Done Sent Done false
15213 4742 Acked Read 8 Done Sent Done false
15214 4743 Acked Write 4 Done Sent Done false
15215 4744 Acked Write 6 Done Sent Done false
15216 4745 Acked Read 6 Done Sent Done false
15217 4746 Acked Read 1 Done Sent Done false
15218 4747 Acked Flush 0 Done Sent Done false
15219 4748 Acked Flush 0 Done Sent Done false
15220 4749 Acked Write 6 Done Sent Done false
15221 4750 Acked Read 1 Done Sent Done false
15222 4751 Acked Read 2 Done Sent Done false
15223 4752 Acked Write 10 Done Sent Done false
15224 4753 Acked Write 8 Done Sent Done false
15225 4754 Acked Write 10 Done Sent Done false
15226 4755 Acked Read 8 Done Sent Done false
15227 4756 Acked Read 4 Done Sent Done false
15228 4757 Acked Read 4 Done Sent Done false
15229 4758 Acked Write 1 Done Sent Done false
15230 4759 Acked Read 10 Done Sent Done false
15231 4760 Acked Read 1 Done Sent Done false
15232 4761 Acked Write 5 Done Sent Done false
15233 4762 Acked Write 2 Done Sent Done false
15234 4763 Acked Write 2 Done Sent Done false
15235 4764 Acked Write 2 Done Sent Done false
15236 4765 Acked Read 9 Done Sent Done false
15237 4766 Acked Write 4 Done Sent Done false
15238 4767 Acked Write 6 Done Sent Done false
15239 4768 Acked Read 4 Done Sent Done false
15240 4769 Acked Write 4 Done Sent Done false
15241 4770 Acked Read 10 Done Sent Done false
15242 4771 Acked Write 7 Done Sent Done false
15243 4772 Acked Write 9 Done Sent Done false
15244 4773 Acked Write 2 Done Sent Done false
15245 4774 Acked Read 1 Done Sent Done false
15246 4775 Acked Read 7 Done Sent Done false
15247 4776 Acked Flush 0 Done Sent Done false
15248 4777 Acked Read 4 Done Sent Done false
15249 4778 Acked Write 8 Done Sent Done false
15250 4779 Acked Read 6 Done Sent Done false
15251 4780 Acked Flush 0 Done Sent Done false
15252 4781 Acked Read 8 Done Sent Done false
15253 4782 Acked Write 8 Done Sent Done false
15254 4783 Acked Read 3 Done Sent Done false
15255 4784 Acked Read 5 Done Sent Done false
15256 4785 Acked Write 1 Done Sent Done false
15257 4786 Acked Flush 0 Done Sent Done false
15258 4787 Acked Write 2 Done Sent Done false
15259 4788 Acked Write 1 Done Sent Done false
15260 4789 Acked Read 1 Done Sent Done false
15261 4790 Acked Write 5 Done Sent Done false
15262 4791 Acked Read 9 Done Sent Done false
15263 4792 Acked Write 10 Done Sent Done false
15264 4793 Acked Write 5 Done Sent Done false
15265 4794 Acked Read 8 Done Sent Done false
15266 4795 Acked Write 7 Done Sent Done false
15267 4796 Acked Read 7 Done Sent Done false
15268 4797 Acked Read 5 Done Sent Done false
15269 4798 Acked Write 10 Done Sent Done false
15270 4799 Acked Read 2 Done Sent Done false
15271 4800 Acked Write 5 Done Sent Done false
15272 4801 Acked Read 9 Done Sent Done false
15273 4802 Acked Read 3 Done Sent Done false
15274 4803 Acked Read 3 Done Sent Done false
15275 4804 Acked Write 10 Done Sent Done false
15276 4805 Acked Write 8 Done Sent Done false
15277 4806 Acked Read 8 Done Sent Done false
15278 4807 Acked Write 1 Done Sent Done false
15279 4808 Acked Flush 0 Done Sent Done false
15280 4809 Acked Write 2 Done Sent Done false
15281 4810 Acked Read 3 Done Sent Done false
15282 4811 Acked Flush 0 Done Sent Done false
15283 4812 Acked Write 4 Done Sent Done false
15284 4813 Acked Write 3 Done Sent Done false
15285 4814 Acked Read 6 Done Sent Done false
15286 4815 Acked Flush 0 Done Sent Done false
15287 4816 Acked Write 7 Done Sent Done false
15288 4817 Acked Flush 0 Done Sent Done false
15289 4818 Acked Write 3 Done Sent Done false
15290 4819 Acked Write 3 Done Sent Done false
15291 4820 Acked Read 10 Done Sent Done false
15292 4821 Acked Write 10 Done Sent Done false
15293 4822 Acked Write 5 Done Sent Done false
15294 4823 Acked Write 9 Done Sent Done false
15295 4824 Acked Read 8 Done Sent Done false
15296 4825 Acked Read 2 Done Sent Done false
15297 4826 Acked Flush 0 Done Sent Done false
15298 4827 Acked Write 3 Done Sent Done false
15299 4828 Acked Read 10 Done Sent Done false
15300 4829 Acked Read 6 Done Sent Done false
15301 4830 Acked Read 10 Done Sent Done false
15302 4831 Acked Flush 0 Done Sent Done false
15303 4832 Acked Write 4 Done Sent Done false
15304 4833 Acked Flush 0 Done Sent Done false
15305 4834 Acked Write 4 Done Sent Done false
15306 4835 Acked Read 4 Done Sent Done false
15307 4836 Acked Write 10 Done Sent Done false
15308 4837 Acked Read 7 Done Sent Done false
15309 4838 Acked Write 1 Done Sent Done false
15310 4839 Acked Read 6 Done Sent Done false
15311 4840 Acked Flush 0 Done Sent Done false
15312 4841 Acked Write 6 Done Sent Done false
15313 4842 Acked Read 9 Done Sent Done false
15314 4843 Acked Write 3 Done Sent Done false
15315 4844 Acked Read 7 Done Sent Done false
15316 4845 Acked Write 4 Done Sent Done false
15317 4846 Acked Write 8 Done Sent Done false
15318 4847 Acked Read 8 Done Sent Done false
15319 4848 Acked Flush 0 Done Sent Done false
15320 4849 Acked Write 3 Done Sent Done false
15321 4850 Acked Read 8 Done Sent Done false
15322 4851 Acked Write 6 Done Sent Done false
15323 4852 Acked Read 10 Done Sent Done false
15324 4853 Acked Read 10 Done Sent Done false
15325 4854 Acked Flush 0 Done Sent Done false
15326 4855 Acked Read 10 Done Sent Done false
15327 4856 Acked Write 9 Done Sent Done false
15328 4857 Acked Flush 0 Done Sent Done false
15329 4858 Acked Read 7 Done Sent Done false
15330 4859 Acked Write 8 Done Sent Done false
15331 4860 Acked Read 9 Done Sent Done false
15332 4861 Acked Read 6 Done Sent Done false
15333 4862 Acked Flush 0 Done Sent Done false
15334 4863 Acked Write 10 Done Sent Done false
15335 4864 Acked Read 3 Done Sent Done false
15336 4865 Acked Write 7 Done Sent Done false
15337 4866 Acked Flush 0 Done Sent Done false
15338 4867 Acked Read 2 Done Sent Done false
15339 4868 Acked Read 7 Done Sent Done false
15340 4869 Acked Write 2 Done Sent Done false
15341 4870 Acked Write 6 Done Sent Done false
15342 4871 Acked Write 1 Done Sent Done false
15343 4872 Acked Write 7 Done Sent Done false
15344 4873 Acked Write 7 Done Sent Done false
15345 4874 Acked Write 1 Done Sent Done false
15346 4875 Acked Write 5 Done Sent Done false
15347 4876 Acked Write 1 Done Sent Done false
15348 4877 Acked Read 1 Done Sent Done false
15349 4878 Acked Write 1 Done Sent Done false
15350 4879 Acked Write 9 Done Sent Done false
15351 4880 Acked Read 1 Done Sent Done false
15352 4881 Acked Write 3 Done Sent Done false
15353 4882 Acked Read 3 Done Sent Done false
15354 4883 Acked Write 5 Done Sent Done false
15355 4884 Acked Write 10 Done Sent Done false
15356 4885 Acked Read 7 Done Sent Done false
15357 4886 Acked Flush 0 Done Sent Done false
15358 4887 Acked Write 8 Done Sent Done false
15359 4888 Acked Flush 0 Done Sent Done false
15360 4889 Acked Read 2 Done Sent Done false
15361 4890 Acked Read 10 Done Sent Done false
15362 4891 Acked Read 2 Done Sent Done false
15363 4892 Acked Read 9 Done Sent Done false
15364 4893 Acked Write 6 Done Sent Done false
15365 4894 Acked Read 3 Done Sent Done false
15366 4895 Acked Read 1 Done Sent Done false
15367 4896 Acked Write 6 Done Sent Done false
15368 4897 Acked Read 5 Done Sent Done false
15369 4898 Acked Write 7 Done Sent Done false
15370 4899 Acked Read 7 Done Sent Done false
15371 4900 Acked Flush 0 Done Sent Done false
15372 4901 Acked Write 1 Done Sent Done false
15373 4902 Acked Read 2 Done Sent Done false
15374 4903 Acked Read 5 Done Sent Done false
15375 4904 Acked Write 1 Done Sent Done false
15376 4905 Acked Read 9 Done Sent Done false
15377 4906 Acked Read 6 Done Sent Done false
15378 4907 Acked Flush 0 Done Sent Done false
15379 4908 Acked Write 1 Done Sent Done false
15380 4909 Acked Write 5 Done Sent Done false
15381 4910 Acked Flush 0 Done Sent Done false
15382 4911 Acked Write 10 Done Sent Done false
15383 4912 Acked Write 5 Done Sent Done false
15384 4913 Acked Write 5 Done Sent Done false
15385 4914 Acked Read 8 Done Sent Done false
15386 4915 Acked Write 6 Done Sent Done false
15387 4916 Acked Write 7 Done Sent Done false
15388 4917 Acked Write 8 Done Sent Done false
15389 4918 Acked Read 9 Done Sent Done false
15390 4919 Acked Write 1 Done Sent Done false
15391 4920 Acked Flush 0 Done Sent Done false
15392 4921 Acked Read 2 Done Sent Done false
15393 4922 Acked Read 2 Done Sent Done false
15394 4923 Acked Flush 0 Done Sent Done false
15395 4924 Acked Read 2 Done Sent Done false
15396 4925 Acked Read 4 Done Sent Done false
15397 4926 Acked Read 10 Done Sent Done false
15398 4927 Acked Write 10 Done Sent Done false
15399 4928 Acked Write 2 Done Sent Done false
15400 4929 Acked Read 7 Done Sent Done false
15401 4930 Acked Write 4 Done Sent Done false
15402 4931 Acked Flush 0 Done Sent Done false
15403 STATES DS:0 DS:1 DS:2 TOTAL
15404 Sent 0 303 0 303
15405 Done 303 0 303 606
15406 Skipped 0 0 0 0
15407 Error 0 0 0 0
15408 Last Flush: 4931 4628 4931
15409 Downstairs last five completed: 4628 4627 4626 4625 4624
15410 Upstairs last five completed: 4931 4930 4929 4928 4927
15411 CLIENT: Up:0 ds:303 act:2
154122024-11-07T14:06:31.367ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154132024-11-07T14:06:31.367ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:56150) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154142024-11-07T14:06:31.368ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154152024-11-07T14:06:31.368ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154162024-11-07T14:06:31.368ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154172024-11-07T14:06:31.368ZINFOcrucible: send last flush ID to this DS: 4628 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154182024-11-07T14:06:31.368ZINFOcrucible: Replied this last flush ID: 4628 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154192024-11-07T14:06:31.368ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154202024-11-07T14:06:31.368ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154212024-11-07T14:06:31.368ZINFOcrucible: [1] client re-new 303 jobs since flush 4628 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154222024-11-07T14:06:31.369ZINFOcrucible: [1] Replayed 303 jobs since flush: 4628 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
15423 ----------------------------------------------------------------
15424 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15425 Downstairs last five completed: 4931 4930 4929 4928 4927
15426 Upstairs last five completed: 4931 4930 4929 4928 4927
15427 CLIENT: Up:0 ds:0 act:3
15428 Replay: All jobs finished
15429 Main task has work to do, go find it
15430 got dsc Stop(1)
15431 stop 1
15432 Nov 07 14:06:33.663 INFO request completed, latency_us: 373, response_code: 204, uri[8820] Got stop action so:false kr:false
15433 : /stop/cid/1, method: POST, req_id: f714a6b3-4407-4347-b493-8f786bc65572, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
154342024-11-07T14:06:33.664ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154352024-11-07T14:06:33.664ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154362024-11-07T14:06:33.664ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154372024-11-07T14:06:33.664ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
154382024-11-07T14:06:33.664ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
15439 Nov 07 14:06:33.664 INFO request completed, latency_us: 145, response_code: 200, uri: /state/cid/1, method: GET, req_id: 49bf0eb8-5690-4a06-a5e7-3cfd8e5baeb4, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
154402024-11-07T14:06:33.664ZINFOcrucible: client task is exiting = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
15441 [8820] Exited with: ExitStatus(unix_wait_status(9))
15442 [8820][1] reports Exit
15443 Nov 07 14:06:37.666 INFO request completed, latency_us: 200, response_code: 200, uri: /state/cid/1, method: GET, req_id: 40f69594-976b-4111-8400-133576e3d84e, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
15444 001/300 Read block 99 len 8192
15445 002/300 Read block 1807 len 28672
15446 003/300 Write block 344 len 8192 data: 4 5
15447 004/300 Read block 63 len 8192
15448 005/300 Read block 137 len 16384
15449 006/300 Read block 1146 len 16384
15450 007/300 Read block 3946 len 8192
15451 008/300 Flush
15452 009/300 Read block 3700 len 12288
15453 010/300 Read block 1843 len 4096
15454 011/300 Write block 4624 len 24576 data: 2 2 2 2 3 4
15455 012/300 Write block 2939 len 24576 data: 3 3 3 4 4 4
15456 013/300 Read block 220 len 24576
15457 014/300 Write block 3754 len 28672 data: 5 3 3 3 3 3 3
15458 015/300 Write block 1786 len 40960 data: 3 3 3 4 4 6 6 5 6 6
15459 016/300 Write block 4619 len 8192 data: 3 4
15460 017/300 Write block 1765 len 4096 data: 3
15461 018/300 Write block 4528 len 36864 data: 7 4 4 4 5 4 5 4 3
15462 019/300 Read block 40 len 40960
15463 020/300 Write block 1896 len 32768 data: 2 2 2 2 2 3 4 4
15464 021/300 Write block 2365 len 4096 data: 5
15465 022/300 Read block 910 len 16384
15466 023/300 Write block 2536 len 40960 data: 4 4 5 9 9 9 8 8 7 6
15467 024/300 Read block 3532 len 20480
15468 025/300 Read block 910 len 12288
15469 026/300 Write block 3257 len 40960 data: 3 3 3 3 3 4 3 3 4 4
15470 027/300 Write block 4730 len 16384 data: 5 5 4 5
15471 028/300 Write block 4076 len 24576 data: 3 3 3 3 3 2
15472 029/300 Read block 1535 len 20480
15473 030/300 Write block 2194 len 12288 data: 3 3 3
15474 031/300 Write block 450 len 20480 data: 5 5 7 5 6
15475 032/300 Write block 2327 len 36864 data: 2 2 2 2 2 2 2 3 4
15476 033/300 Read block 3663 len 40960
15477 034/300 Write block 3650 len 20480 data: 3 3 2 2 3
15478 035/300 Write block 3317 len 8192 data: 2 2
15479 036/300 Read block 4653 len 12288
15480 037/300 Flush
15481 038/300 Read block 57 len 8192
15482 039/300 Flush
15483 040/300 Read block 4496 len 24576
15484 041/300 Flush
15485 042/300 Read block 713 len 20480
15486 043/300 Read block 2438 len 12288
15487 044/300 Flush
15488 045/300 Write block 4345 len 36864 data: 3 3 4 5 5 5 5 5 4
15489 046/300 Write block 3017 len 12288 data: 4 3 3
15490 047/300 Write block 3017 len 36864 data: 5 4 4 3 3 3 3 3 3
15491 048/300 Flush
15492 049/300 Read block 122 len 12288
15493 050/300 Read block 899 len 12288
15494 051/300 Read block 4356 len 32768
15495 052/300 Write block 4918 len 36864 data: 5 6 6 4 4 3 3 3 4
15496 053/300 Read block 4099 len 8192
15497 054/300 Write block 4697 len 4096 data: 7
15498 055/300 Read block 1365 len 36864
15499 056/300 Read block 1534 len 36864
15500 057/300 Read block 503 len 24576
15501 058/300 Read block 2665 len 40960
15502 059/300 Write block 3125 len 28672 data: 3 3 3 3 2 4 4
15503 060/300 Read block 2947 len 16384
15504 061/300 Read block 3027 len 4096
15505 062/300 Read block 691 len 36864
15506 063/300 Write block 2752 len 24576 data: 3 3 3 3 3 3
15507 064/300 Read block 2556 len 12288
15508 065/300 Write block 42 len 36864 data: 3 3 3 3 3 3 4 4 4
15509 066/300 Write block 3511 len 16384 data: 6 5 4 4
15510 067/300 Read block 3760 len 24576
15511 068/300 Write block 1338 len 24576 data: 5 3 3 3 3 3
15512 069/300 Flush
15513 070/300 Read block 1984 len 36864
15514 071/300 Read block 2017 len 36864
15515 072/300 Read block 3427 len 24576
15516 073/300 Write block 776 len 12288 data: 3 2 3
15517 074/300 Read block 2347 len 20480
15518 075/300 Read block 3416 len 4096
15519 076/300 Write block 4610 len 8192 data: 3 3
15520 077/300 Write block 4729 len 28672 data: 6 6 6 5 6 4 4
15521 078/300 Read block 2420 len 4096
15522 079/300 Read block 1670 len 24576
15523 080/300 Write block 3965 len 20480 data: 4 3 3 3 3
15524 081/300 Read block 4814 len 28672
15525 082/300 Read block 4484 len 40960
15526 083/300 Read block 2511 len 8192
15527 084/300 Flush
15528 085/300 Flush
15529 086/300 Read block 1447 len 20480
15530 087/300 Read block 756 len 32768
15531 088/300 Read block 1213 len 32768
15532 089/300 Write block 2750 len 8192 data: 3 2
15533 090/300 Read block 1445 len 12288
15534 091/300 Write block 3305 len 32768 data: 3 3 3 3 3 3 3 3
15535 092/300 Write block 4778 len 12288 data: 3 4 4
15536 093/300 Read block 4573 len 40960
15537 094/300 Read block 925 len 40960
15538 095/300 Read block 4320 len 12288
15539 096/300 Read block 1530 len 36864
15540 097/300 Write block 4965 len 4096 data: 4
15541 098/300 Write block 3465 len 28672 data: 4 3 3 3 3 3 3
15542 099/300 Write block 4640 len 40960 data: 2 3 3 3 3 3 3 4 4 3
15543 100/300 Write block 3858 len 12288 data: 2 3 4
15544 101/300 Read block 3954 len 4096
15545 102/300 Write block 1012 len 20480 data: 2 2 3 4 3
15546 103/300 Write block 3715 len 24576 data: 4 4 5 5 4 4
15547 104/300 Write block 829 len 20480 data: 4 4 4 4 4
15548 105/300 Write block 4798 len 24576 data: 3 4 4 4 5 5
15549 106/300 Read block 4945 len 40960
15550 107/300 Write block 4309 len 16384 data: 3 3 5 4
15551 108/300 Read block 927 len 20480
15552 109/300 Read block 4377 len 32768
15553 110/300 Read block 4787 len 24576
15554 111/300 Write block 1835 len 4096 data: 3
15555 112/300 Read block 3066 len 24576
15556 113/300 Read block 847 len 32768
15557 114/300 Read block 4676 len 4096
15558 115/300 Write block 1154 len 24576 data: 4 4 4 3 3 4
15559 116/300 Write block 1097 len 12288 data: 3 4 4
15560 117/300 Read block 2522 len 36864
15561 118/300 Write block 239 len 8192 data: 4 6
15562 119/300 Write block 155 len 12288 data: 4 4 5
15563 120/300 Write block 3628 len 28672 data: 5 6 7 5 5 3 3
15564 121/300 Read block 3609 len 36864
15565 122/300 Write block 2596 len 4096 data: 5
15566 123/300 Write block 2921 len 8192 data: 3 2
15567 124/300 Read block 3537 len 28672
15568 125/300 Read block 2004 len 36864
15569 126/300 Write block 821 len 8192 data: 4 4
15570 127/300 Read block 1907 len 36864
15571 128/300 Write block 4639 len 20480 data: 3 3 4 4 4
15572 129/300 Read block 3302 len 20480
15573 130/300 Flush
15574 131/300 Write block 1182 len 32768 data: 2 2 2 2 3 4 5 5
15575 132/300 Write block 824 len 28672 data: 5 5 4 2 4 5 5
15576 133/300 Write block 1412 len 28672 data: 2 3 3 3 3 3 3
15577 134/300 Read block 2282 len 12288
15578 135/300 Read block 259 len 24576
15579 136/300 Write block 4441 len 40960 data: 6 6 5 5 5 4 4 4 4 4
15580 137/300 Read block 294 len 20480
15581 138/300 Read block 2739 len 32768
15582 139/300 Write block 2356 len 4096 data: 2
15583 140/300 Read block 1404 len 20480
15584 141/300 Read block 4270 len 16384
15585 142/300 Write block 3200 len 36864 data: 3 3 3 3 3 3 3 3 4
15586 143/300 Read block 3956 len 40960
15587 144/300 Read block 3667 len 32768
15588 145/300 Write block 623 len 16384 data: 3 2 2 3
15589 146/300 Read block 937 len 36864
15590 147/300 Read block 1744 len 36864
15591 148/300 Read block 4929 len 8192
15592 149/300 Read block 899 len 24576
15593 150/300 Write block 1403 len 20480 data: 4 4 4 4 4
15594 151/300 Write block 2936 len 16384 data: 3 4 3 4
15595 152/300 Read block 4102 len 8192
15596 153/300 Write block 2635 len 36864 data: 3 2 2 2 2 3 4 5 5
15597 154/300 Write block 481 len 32768 data: 5 6 5 5 6 5 5 5
15598 155/300 Write block 2997 len 4096 data: 4
15599 156/300 Read block 4979 len 28672
15600 157/300 Read block 2500 len 32768
15601 158/300 Write block 2700 len 20480 data: 5 3 3 3 2
15602 159/300 Flush
15603 160/300 Read block 1393 len 36864
15604 161/300 Write block 4200 len 32768 data: 5 5 5 5 6 4 4 3
15605 162/300 Write block 609 len 32768 data: 2 3 3 3 3 3 2 2
15606 163/300 Write block 3869 len 12288 data: 4 5 4
15607 164/300 Read block 901 len 40960
15608 165/300 Read block 417 len 4096
15609 166/300 Read block 3218 len 24576
15610 167/300 Read block 1079 len 16384
15611 168/300 Flush
15612 169/300 Write block 312 len 24576 data: 3 3 3 2 2 2
15613 170/300 Flush
15614 171/300 Write block 1880 len 8192 data: 4 4
15615 172/300 Read block 4249 len 8192
15616 173/300 Read block 405 len 24576
15617 174/300 Write block 2589 len 28672 data: 4 4 5 5 5 5 5
15618 175/300 Write block 3450 len 4096 data: 3
15619 176/300 Flush
15620 177/300 Write block 2308 len 20480 data: 5 4 2 3 3
15621 178/300 Read block 2427 len 28672
15622 179/300 Write block 2974 len 28672 data: 3 3 5 5 5 4 4
15623 180/300 Flush
15624 181/300 Read block 1807 len 28672
15625 182/300 Write block 894 len 40960 data: 6 6 8 6 7 8 7 6 5 7
15626 183/300 Read block 2099 len 28672
15627 184/300 Read block 875 len 24576
15628 185/300 Read block 3458 len 24576
15629 186/300 Write block 975 len 28672 data: 3 4 4 4 3 5 6
15630 187/300 Write block 2945 len 28672 data: 4 4 4 4 3 3 3
15631 188/300 Read block 4255 len 8192
15632 189/300 Write block 1599 len 36864 data: 5 6 6 6 6 6 5 4 3
15633 190/300 Read block 4021 len 12288
15634 191/300 Write block 3590 len 4096 data: 5
15635 192/300 Write block 439 len 32768 data: 6 6 4 5 4 6 5 4
15636 193/300 Read block 1646 len 4096
15637 194/300 Read block 3972 len 36864
15638 195/300 Write block 4058 len 40960 data: 2 2 3 4 4 3 3 2 2 2
15639 196/300 Read block 807 len 16384
15640 197/300 Read block 1455 len 8192
15641 198/300 Read block 3473 len 32768
15642 199/300 Write block 805 len 12288 data: 2 2 2
15643 200/300 Read block 2115 len 24576
15644 201/300 Read block 2817 len 16384
15645 202/300 Read block 2740 len 8192
15646 203/300 Write block 740 len 28672 data: 4 4 3 4 4 5 4
15647 204/300 Read block 601 len 32768
15648 205/300 Flush
15649 206/300 Read block 2711 len 36864
15650 207/300 Read block 3212 len 40960
15651 208/300 Read block 1598 len 8192
15652 209/300 Read block 3304 len 16384
15653 210/300 Read block 1935 len 40960
15654 211/300 Read block 2449 len 20480
15655 212/300 Write block 1467 len 36864 data: 3 3 2 2 2 2 3 3 3
15656 213/300 Flush
15657 214/300 Write block 3536 len 32768 data: 4 4 4 4 2 2 3 3
15658 215/300 Read block 4653 len 16384
15659 216/300 Write block 1396 len 36864 data: 2 2 2 3 3 4 4 5 5
15660 217/300 Flush
15661 218/300 Flush
15662 219/300 Write block 4530 len 36864 data: 5 5 6 5 6 5 4 4 4
15663 220/300 Write block 4360 len 28672 data: 3 3 4 3 3 3 5
15664 221/300 Write block 141 len 24576 data: 3 3 3 3 3 4
15665 222/300 Write block 2612 len 4096 data: 4
15666 223/300 Read block 4594 len 4096
15667 224/300 Write block 4167 len 40960 data: 2 2 2 2 4 4 5 6 5 5
15668 225/300 Read block 2366 len 4096
15669 226/300 Write block 3046 len 12288 data: 2 3 4
15670 227/300 Write block 3718 len 40960 data: 6 5 5 4 4 3 3 3 3 2
15671 228/300 Read block 3571 len 12288
15672 229/300 Read block 3739 len 28672
15673 230/300 Read block 4728 len 8192
15674 231/300 Write block 2445 len 8192 data: 4 5
15675 232/300 Read block 1688 len 40960
15676 233/300 Read block 3134 len 36864
15677 234/300 Read block 1754 len 24576
15678 235/300 Read block 4846 len 8192
15679 236/300 Write block 4635 len 16384 data: 4 4 3 3
15680 237/300 Write block 3412 len 20480 data: 3 3 3 3 3
15681 238/300 Write block 647 len 20480 data: 5 6 5 5 6
15682 239/300 Write block 2482 len 40960 data: 7 7 5 6 4 4 4 3 3 6
15683 240/300 Write block 1681 len 24576 data: 5 4 3 2 2 2
15684 241/300 Read block 2157 len 4096
15685 242/300 Read block 3234 len 20480
15686 243/300 Read block 1282 len 20480
15687 244/300 Write block 3923 len 36864 data: 5 7 5 5 6 6 3 3 3
15688 245/300 Write block 954 len 24576 data: 6 5 6 5 3 2
15689 246/300 Read block 1464 len 40960
15690 247/300 Read block 3824 len 36864
15691 248/300 Write block 1423 len 16384 data: 2 2 3 3
15692 249/300 Read block 4090 len 20480
15693 250/300 Write block 3022 len 32768 data: 4 4 4 4 3 3 4 4
15694 251/300 Write block 222 len 4096 data: 2
15695 252/300 Read block 2587 len 8192
15696 253/300 Flush
15697 254/300 Read block 4764 len 24576
15698 255/300 Read block 1294 len 4096
15699 256/300 Read block 4993 len 16384
15700 257/300 Write block 4121 len 28672 data: 5 5 5 5 4 4 4
15701 258/300 Read block 2334 len 28672
15702 259/300 Write block 404 len 36864 data: 6 6 4 3 3 3 4 4 3
15703 260/300 Write block 644 len 12288 data: 4 5 5
15704 261/300 Write block 2593 len 28672 data: 6 6 6 6 5 4 4
15705 262/300 Flush
15706 263/300 Read block 2718 len 4096
15707 264/300 Write block 2789 len 32768 data: 5 5 5 5 5 5 3 3
15708 265/300 Read block 1139 len 36864
15709 266/300 Read block 95 len 24576
15710 267/300 Read block 992 len 40960
15711 268/300 Write block 4306 len 40960 data: 2 2 2 4 4 6 5 5 7 6
15712 269/300 Read block 4935 len 20480
15713 270/300 Read block 2913 len 16384
15714 271/300 Read block 1625 len 24576
15715 272/300 Write block 328 len 40960 data: 4 4 4 3 3 4 3 4 4 3
15716 273/300 Read block 2996 len 12288
15717 274/300 Write block 2981 len 40960 data: 3 3 3 3 3 3 3 3 3 3
15718 275/300 Read block 3112 len 24576
15719 276/300 Write block 82 len 20480 data: 4 4 4 4 4
15720 277/300 Read block 1467 len 20480
15721 278/300 Flush
15722 279/300 Write block 2906 len 36864 data: 3 3 3 4 5 6 6 5 5
15723 280/300 Write block 986 len 12288 data: 4 4 4
15724 281/300 Read block 1853 len 20480
15725 282/300 Write block 1497 len 36864 data: 3 3 3 2 2 3 3 3 4
15726 283/300 Read block 4658 len 8192
15727 284/300 Flush
15728 285/300 Write block 265 len 28672 data: 3 3 2 2 2 2 2
15729 286/300 Read block 3439 len 12288
15730 287/300 Read block 218 len 36864
15731 288/300 Read block 4929 len 12288
15732 289/300 Flush
15733 290/300 Write block 1242 len 24576 data: 3 3 2 2 2 3
15734 291/300 Read block 3034 len 4096
15735 292/300 Read block 1046 len 32768
15736 293/300 Read block 3881 len 12288
15737 294/300 Write block 2951 len 28672 data: 4 3 3 3 3 2 3
15738 295/300 Write block 4020 len 4096 data: 3
15739 296/300 Write block 3636 len 24576 data: 2 3 3 3 3 3
15740 297/300 Read block 970 len 28672
15741 298/300 Read block 4116 len 8192
15742 299/300 Read block 808 len 8192
15743 300/300 Write block 1773 len 12288 data: 5 5 5
15744 301/300 Write block 4000 len 36864 data: 4 4 4 4 4 3 4 5 4
15745 Main task has work to do, go find it
15746 got dsc Start(1)
15747 start 1
15748 Nov 07 14:06:37.955 INFO request completed, latency_us: 215, response_code: 204, uri: /start/cid/1, method: POST, req_id: bdcc9988-f7cf-422b-8e13-c677c79dec6c, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
15749 Replay: started 1, returned:Ok(())
15750 ----------------------------------------------------------------
15751 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15752 States: Active Offline Active
15753 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15754 4932 Acked Read 2 Done Sent Done false
15755 4933 Acked Read 7 Done Sent Done false
15756 4934 Acked Write 2 Done Sent Done false
15757 4935 Acked Read 2 Done Sent Done false
15758 4936 Acked Read 4 Done Sent Done false
15759 4937 Acked Read 4 Done Sent Done false
15760 4938 Acked Read 2 Done Sent Done false
15761 4939 Acked Flush 0 Done Sent Done false
15762 4940 Acked Read 3 Done Sent Done false
15763 4941 Acked Read 1 Done Sent Done false
15764 4942 Acked Write 6 Done Sent Done false
15765 4943 Acked Write 6 Done Sent Done false
15766 4944 Acked Read 6 Done Sent Done false
15767 4945 Acked Write 7 Done Sent Done false
15768 4946 Acked Write 10 Done Sent Done false
15769 4947 Acked Write 2 Done Sent Done false
15770 4948 Acked Write 1 Done Sent Done false
15771 4949 Acked Write 9 Done Sent Done false
15772 4950 Acked Read 10 Done Sent Done false
15773 4951 Acked Write 8 Done Sent Done false
15774 4952 Acked Write 1 Done Sent Done false
15775 4953 Acked Read 4 Done Sent Done false
15776 4954 Acked Write 10 Done Sent Done false
15777 4955 Acked Read 5 Done Sent Done false
15778 4956 Acked Read 3 Done Sent Done false
15779 4957 Acked Write 10 Done Sent Done false
15780 4958 Acked Write 4 Done Sent Done false
15781 4959 Acked Write 6 Done Sent Done false
15782 4960 Acked Read 5 Done Sent Done false
15783 4961 Acked Write 3 Done Sent Done false
15784 4962 Acked Write 5 Done Sent Done false
15785 4963 Acked Write 9 Done Sent Done false
15786 4964 Acked Read 10 Done Sent Done false
15787 4965 Acked Write 5 Done Sent Done false
15788 4966 Acked Write 2 Done Sent Done false
15789 4967 Acked Read 3 Done Sent Done false
15790 4968 Acked Flush 0 Done Sent Done false
15791 4969 Acked Read 2 Done Sent Done false
15792 4970 Acked Flush 0 Done Sent Done false
15793 4971 Acked Read 6 Done Sent Done false
15794 4972 Acked Flush 0 Done Sent Done false
15795 4973 Acked Read 5 Done Sent Done false
15796 4974 Acked Read 3 Done Sent Done false
15797 4975 Acked Flush 0 Done Sent Done false
15798 4976 Acked Write 9 Done Sent Done false
15799 4977 Acked Write 3 Done Sent Done false
15800 4978 Acked Write 9 Done Sent Done false
15801 4979 Acked Flush 0 Done Sent Done false
15802 4980 Acked Read 3 Done Sent Done false
15803 4981 Acked Read 3 Done Sent Done false
15804 4982 Acked Read 8 Done Sent Done false
15805 4983 Acked Write 9 Done Sent Done false
15806 4984 Acked Read 2 Done Sent Done false
15807 4985 Acked Write 1 Done Sent Done false
15808 4986 Acked Read 9 Done Sent Done false
15809 4987 Acked Read 9 Done Sent Done false
15810 4988 Acked Read 6 Done Sent Done false
15811 4989 Acked Read 10 Done Sent Done false
15812 4990 Acked Write 7 Done Sent Done false
15813 4991 Acked Read 4 Done Sent Done false
15814 4992 Acked Read 1 Done Sent Done false
15815 4993 Acked Read 9 Done Sent Done false
15816 4994 Acked Write 6 Done Sent Done false
15817 4995 Acked Read 3 Done Sent Done false
15818 4996 Acked Write 9 Done Sent Done false
15819 4997 Acked Write 4 Done Sent Done false
15820 4998 Acked Read 6 Done Sent Done false
15821 4999 Acked Write 6 Done Sent Done false
15822 5000 Acked Flush 0 Done Sent Done false
15823 5001 Acked Read 9 Done Sent Done false
15824 5002 Acked Read 9 Done Sent Done false
15825 5003 Acked Read 6 Done Sent Done false
15826 5004 Acked Write 3 Done Sent Done false
15827 5005 Acked Read 5 Done Sent Done false
15828 5006 Acked Read 1 Done Sent Done false
15829 5007 Acked Write 2 Done Sent Done false
15830 5008 Acked Write 7 Done Sent Done false
15831 5009 Acked Read 1 Done Sent Done false
15832 5010 Acked Read 6 Done Sent Done false
15833 5011 Acked Write 5 Done Sent Done false
15834 5012 Acked Read 7 Done Sent Done false
15835 5013 Acked Read 10 Done Sent Done false
15836 5014 Acked Read 2 Done Sent Done false
15837 5015 Acked Flush 0 Done Sent Done false
15838 5016 Acked Flush 0 Done Sent Done false
15839 5017 Acked Read 5 Done Sent Done false
15840 5018 Acked Read 8 Done Sent Done false
15841 5019 Acked Read 8 Done Sent Done false
15842 5020 Acked Write 2 Done Sent Done false
15843 5021 Acked Read 3 Done Sent Done false
15844 5022 Acked Write 8 Done Sent Done false
15845 5023 Acked Write 3 Done Sent Done false
15846 5024 Acked Read 10 Done Sent Done false
15847 5025 Acked Read 10 Done Sent Done false
15848 5026 Acked Read 3 Done Sent Done false
15849 5027 Acked Read 9 Done Sent Done false
15850 5028 Acked Write 1 Done Sent Done false
15851 5029 Acked Write 7 Done Sent Done false
15852 5030 Acked Write 10 Done Sent Done false
15853 5031 Acked Write 3 Done Sent Done false
15854 5032 Acked Read 1 Done Sent Done false
15855 5033 Acked Write 5 Done Sent Done false
15856 5034 Acked Write 6 Done Sent Done false
15857 5035 Acked Write 5 Done Sent Done false
15858 5036 Acked Write 6 Done Sent Done false
15859 5037 Acked Read 10 Done Sent Done false
15860 5038 Acked Write 4 Done Sent Done false
15861 5039 Acked Read 5 Done Sent Done false
15862 5040 Acked Read 8 Done Sent Done false
15863 5041 Acked Read 6 Done Sent Done false
15864 5042 Acked Write 1 Done Sent Done false
15865 5043 Acked Read 6 Done Sent Done false
15866 5044 Acked Read 8 Done Sent Done false
15867 5045 Acked Read 1 Done Sent Done false
15868 5046 Acked Write 6 Done Sent Done false
15869 5047 Acked Write 3 Done Sent Done false
15870 5048 Acked Read 9 Done Sent Done false
15871 5049 Acked Write 2 Done Sent Done false
15872 5050 Acked Write 3 Done Sent Done false
15873 5051 Acked Write 7 Done Sent Done false
15874 5052 Acked Read 9 Done Sent Done false
15875 5053 Acked Write 1 Done Sent Done false
15876 5054 Acked Write 2 Done Sent Done false
15877 5055 Acked Read 7 Done Sent Done false
15878 5056 Acked Read 9 Done Sent Done false
15879 5057 Acked Write 2 Done Sent Done false
15880 5058 Acked Read 9 Done Sent Done false
15881 5059 Acked Write 5 Done Sent Done false
15882 5060 Acked Read 5 Done Sent Done false
15883 5061 Acked Flush 0 Done Sent Done false
15884 5062 Acked Write 8 Done Sent Done false
15885 5063 Acked Write 7 Done Sent Done false
15886 5064 Acked Write 7 Done Sent Done false
15887 5065 Acked Read 3 Done Sent Done false
15888 5066 Acked Read 6 Done Sent Done false
15889 5067 Acked Write 10 Done Sent Done false
15890 5068 Acked Read 5 Done Sent Done false
15891 5069 Acked Read 8 Done Sent Done false
15892 5070 Acked Write 1 Done Sent Done false
15893 5071 Acked Read 5 Done Sent Done false
15894 5072 Acked Read 4 Done Sent Done false
15895 5073 Acked Write 9 Done Sent Done false
15896 5074 Acked Read 10 Done Sent Done false
15897 5075 Acked Read 8 Done Sent Done false
15898 5076 Acked Write 4 Done Sent Done false
15899 5077 Acked Read 9 Done Sent Done false
15900 5078 Acked Read 9 Done Sent Done false
15901 5079 Acked Read 2 Done Sent Done false
15902 5080 Acked Read 6 Done Sent Done false
15903 5081 Acked Write 5 Done Sent Done false
15904 5082 Acked Write 4 Done Sent Done false
15905 5083 Acked Read 2 Done Sent Done false
15906 5084 Acked Write 9 Done Sent Done false
15907 5085 Acked Write 8 Done Sent Done false
15908 5086 Acked Write 1 Done Sent Done false
15909 5087 Acked Read 7 Done Sent Done false
15910 5088 Acked Read 8 Done Sent Done false
15911 5089 Acked Write 5 Done Sent Done false
15912 5090 Acked Flush 0 Done Sent Done false
15913 5091 Acked Read 9 Done Sent Done false
15914 5092 Acked Write 8 Done Sent Done false
15915 5093 Acked Write 8 Done Sent Done false
15916 5094 Acked Write 3 Done Sent Done false
15917 5095 Acked Read 10 Done Sent Done false
15918 5096 Acked Read 1 Done Sent Done false
15919 5097 Acked Read 6 Done Sent Done false
15920 5098 Acked Read 4 Done Sent Done false
15921 5099 Acked Flush 0 Done Sent Done false
15922 5100 Acked Write 6 Done Sent Done false
15923 5101 Acked Flush 0 Done Sent Done false
15924 5102 Acked Write 2 Done Sent Done false
15925 5103 Acked Read 2 Done Sent Done false
15926 5104 Acked Read 6 Done Sent Done false
15927 5105 Acked Write 7 Done Sent Done false
15928 5106 Acked Write 1 Done Sent Done false
15929 5107 Acked Flush 0 Done Sent Done false
15930 5108 Acked Write 5 Done Sent Done false
15931 5109 Acked Read 7 Done Sent Done false
15932 5110 Acked Write 7 Done Sent Done false
15933 5111 Acked Flush 0 Done Sent Done false
15934 5112 Acked Read 7 Done Sent Done false
15935 5113 Acked Write 10 Done Sent Done false
15936 5114 Acked Read 7 Done Sent Done false
15937 5115 Acked Read 6 Done Sent Done false
15938 5116 Acked Read 6 Done Sent Done false
15939 5117 Acked Write 7 Done Sent Done false
15940 5118 Acked Write 7 Done Sent Done false
15941 5119 Acked Read 2 Done Sent Done false
15942 5120 Acked Write 9 Done Sent Done false
15943 5121 Acked Read 3 Done Sent Done false
15944 5122 Acked Write 1 Done Sent Done false
15945 5123 Acked Write 8 Done Sent Done false
15946 5124 Acked Read 1 Done Sent Done false
15947 5125 Acked Read 9 Done Sent Done false
15948 5126 Acked Write 10 Done Sent Done false
15949 5127 Acked Read 4 Done Sent Done false
15950 5128 Acked Read 2 Done Sent Done false
15951 5129 Acked Read 8 Done Sent Done false
15952 5130 Acked Write 3 Done Sent Done false
15953 5131 Acked Read 6 Done Sent Done false
15954 5132 Acked Read 4 Done Sent Done false
15955 5133 Acked Read 2 Done Sent Done false
15956 5134 Acked Write 7 Done Sent Done false
15957 5135 Acked Read 8 Done Sent Done false
15958 5136 Acked Flush 0 Done Sent Done false
15959 5137 Acked Read 9 Done Sent Done false
15960 5138 Acked Read 10 Done Sent Done false
15961 5139 Acked Read 2 Done Sent Done false
15962 5140 Acked Read 4 Done Sent Done false
15963 5141 Acked Read 10 Done Sent Done false
15964 5142 Acked Read 5 Done Sent Done false
15965 5143 Acked Write 9 Done Sent Done false
15966 5144 Acked Flush 0 Done Sent Done false
15967 5145 Acked Write 8 Done Sent Done false
15968 5146 Acked Read 4 Done Sent Done false
15969 5147 Acked Write 9 Done Sent Done false
15970 5148 Acked Flush 0 Done Sent Done false
15971 5149 Acked Flush 0 Done Sent Done false
15972 5150 Acked Write 9 Done Sent Done false
15973 5151 Acked Write 7 Done Sent Done false
15974 5152 Acked Write 6 Done Sent Done false
15975 5153 Acked Write 1 Done Sent Done false
15976 5154 Acked Read 1 Done Sent Done false
15977 5155 Acked Write 10 Done Sent Done false
15978 5156 Acked Read 1 Done Sent Done false
15979 5157 Acked Write 3 Done Sent Done false
15980 5158 Acked Write 10 Done Sent Done false
15981 5159 Acked Read 3 Done Sent Done false
15982 5160 Acked Read 7 Done Sent Done false
15983 5161 Acked Read 2 Done Sent Done false
15984 5162 Acked Write 2 Done Sent Done false
15985 5163 Acked Read 10 Done Sent Done false
15986 5164 Acked Read 9 Done Sent Done false
15987 5165 Acked Read 6 Done Sent Done false
15988 5166 Acked Read 2 Done Sent Done false
15989 5167 Acked Write 4 Done Sent Done false
15990 5168 Acked Write 5 Done Sent Done false
15991 5169 Acked Write 5 Done Sent Done false
15992 5170 Acked Write 10 Done Sent Done false
15993 5171 Acked Write 6 Done Sent Done false
15994 5172 Acked Read 1 Done Sent Done false
15995 5173 Acked Read 5 Done Sent Done false
15996 5174 Acked Read 5 Done Sent Done false
15997 5175 Acked Write 9 Done Sent Done false
15998 5176 Acked Write 6 Done Sent Done false
15999 5177 Acked Read 10 Done Sent Done false
16000 5178 Acked Read 9 Done Sent Done false
16001 5179 Acked Write 4 Done Sent Done false
16002 5180 Acked Read 5 Done Sent Done false
16003 5181 Acked Write 8 Done Sent Done false
16004 5182 Acked Write 1 Done Sent Done false
16005 5183 Acked Read 2 Done Sent Done false
16006 5184 Acked Flush 0 Done Sent Done false
16007 5185 Acked Read 6 Done Sent Done false
16008 5186 Acked Read 1 Done Sent Done false
16009 5187 Acked Read 4 Done Sent Done false
16010 5188 Acked Write 7 Done Sent Done false
16011 5189 Acked Read 7 Done Sent Done false
16012 5190 Acked Write 9 Done Sent Done false
16013 5191 Acked Write 3 Done Sent Done false
16014 5192 Acked Write 7 Done Sent Done false
16015 5193 Acked Flush 0 Done Sent Done false
16016 5194 Acked Read 1 Done Sent Done false
16017 5195 Acked Write 8 Done Sent Done false
16018 5196 Acked Read 9 Done Sent Done false
16019 5197 Acked Read 6 Done Sent Done false
16020 5198 Acked Read 10 Done Sent Done false
16021 5199 Acked Write 10 Done Sent Done false
16022 5200 Acked Read 5 Done Sent Done false
16023 5201 Acked Read 4 Done Sent Done false
16024 5202 Acked Read 6 Done Sent Done false
16025 5203 Acked Write 10 Done Sent Done false
16026 5204 Acked Read 3 Done Sent Done false
16027 5205 Acked Write 10 Done Sent Done false
16028 5206 Acked Read 6 Done Sent Done false
16029 5207 Acked Write 5 Done Sent Done false
16030 5208 Acked Read 5 Done Sent Done false
16031 5209 Acked Flush 0 Done Sent Done false
16032 5210 Acked Write 9 Done Sent Done false
16033 5211 Acked Write 3 Done Sent Done false
16034 5212 Acked Read 5 Done Sent Done false
16035 5213 Acked Write 9 Done Sent Done false
16036 5214 Acked Read 2 Done Sent Done false
16037 5215 Acked Flush 0 Done Sent Done false
16038 5216 Acked Write 7 Done Sent Done false
16039 5217 Acked Read 3 Done Sent Done false
16040 5218 Acked Read 9 Done Sent Done false
16041 5219 Acked Read 3 Done Sent Done false
16042 5220 Acked Flush 0 Done Sent Done false
16043 5221 Acked Write 6 Done Sent Done false
16044 5222 Acked Read 1 Done Sent Done false
16045 5223 Acked Read 8 Done Sent Done false
16046 5224 Acked Read 3 Done Sent Done false
16047 5225 Acked Write 7 Done Sent Done false
16048 5226 Acked Write 1 Done Sent Done false
16049 5227 Acked Write 6 Done Sent Done false
16050 5228 Acked Read 7 Done Sent Done false
16051 5229 Acked Read 2 Done Sent Done false
16052 5230 Acked Read 2 Done Sent Done false
16053 5231 Acked Write 3 Done Sent Done false
16054 5232 Acked Write 9 Done Sent Done false
16055 STATES DS:0 DS:1 DS:2 TOTAL
16056 Sent 0 301 0 301
16057 Done 301 0 301 602
16058 Skipped 0 0 0 0
16059 Error 0 0 0 0
16060 Last Flush: 5220 4931 5220
16061 Downstairs last five completed: 4931 4930 4929 4928 4927
16062 Upstairs last five completed: 5232 5231 5230 5229 5228
16063 CLIENT: Up:0 ds:301 act:2
16064 [8820] got start action
16065 [8820] I am going to restart
16066 Starting downstairs at port 8820
16067 Make output file at "/tmp/dsc/downstairs-8820.txt"
16068 [8820][1] reports Starting
16069 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1213)
16070 [8820][1] reports Running
16071 ----------------------------------------------------------------
16072 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16073 States: Active Offline Active
16074 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16075 4932 Acked Read 2 Done Sent Done false
16076 4933 Acked Read 7 Done Sent Done false
16077 4934 Acked Write 2 Done Sent Done false
16078 4935 Acked Read 2 Done Sent Done false
16079 4936 Acked Read 4 Done Sent Done false
16080 4937 Acked Read 4 Done Sent Done false
16081 4938 Acked Read 2 Done Sent Done false
16082 4939 Acked Flush 0 Done Sent Done false
16083 4940 Acked Read 3 Done Sent Done false
16084 4941 Acked Read 1 Done Sent Done false
16085 4942 Acked Write 6 Done Sent Done false
16086 4943 Acked Write 6 Done Sent Done false
16087 4944 Acked Read 6 Done Sent Done false
16088 4945 Acked Write 7 Done Sent Done false
16089 4946 Acked Write 10 Done Sent Done false
16090 4947 Acked Write 2 Done Sent Done false
16091 4948 Acked Write 1 Done Sent Done false
16092 4949 Acked Write 9 Done Sent Done false
16093 4950 Acked Read 10 Done Sent Done false
16094 4951 Acked Write 8 Done Sent Done false
16095 4952 Acked Write 1 Done Sent Done false
16096 4953 Acked Read 4 Done Sent Done false
16097 4954 Acked Write 10 Done Sent Done false
16098 4955 Acked Read 5 Done Sent Done false
16099 4956 Acked Read 3 Done Sent Done false
16100 4957 Acked Write 10 Done Sent Done false
16101 4958 Acked Write 4 Done Sent Done false
16102 4959 Acked Write 6 Done Sent Done false
16103 4960 Acked Read 5 Done Sent Done false
16104 4961 Acked Write 3 Done Sent Done false
16105 4962 Acked Write 5 Done Sent Done false
16106 4963 Acked Write 9 Done Sent Done false
16107 4964 Acked Read 10 Done Sent Done false
16108 4965 Acked Write 5 Done Sent Done false
16109 4966 Acked Write 2 Done Sent Done false
16110 4967 Acked Read 3 Done Sent Done false
16111 4968 Acked Flush 0 Done Sent Done false
16112 4969 Acked Read 2 Done Sent Done false
16113 4970 Acked Flush 0 Done Sent Done false
16114 4971 Acked Read 6 Done Sent Done false
16115 4972 Acked Flush 0 Done Sent Done false
16116 4973 Acked Read 5 Done Sent Done false
16117 4974 Acked Read 3 Done Sent Done false
16118 4975 Acked Flush 0 Done Sent Done false
16119 4976 Acked Write 9 Done Sent Done false
16120 4977 Acked Write 3 Done Sent Done false
16121 4978 Acked Write 9 Done Sent Done false
16122 4979 Acked Flush 0 Done Sent Done false
16123 4980 Acked Read 3 Done Sent Done false
16124 4981 Acked Read 3 Done Sent Done false
16125 4982 Acked Read 8 Done Sent Done false
16126 4983 Acked Write 9 Done Sent Done false
16127 4984 Acked Read 2 Done Sent Done false
16128 4985 Acked Write 1 Done Sent Done false
16129 4986 Acked Read 9 Done Sent Done false
16130 4987 Acked Read 9 Done Sent Done false
16131 4988 Acked Read 6 Done Sent Done false
16132 4989 Acked Read 10 Done Sent Done false
16133 4990 Acked Write 7 Done Sent Done false
16134 4991 Acked Read 4 Done Sent Done false
16135 4992 Acked Read 1 Done Sent Done false
16136 4993 Acked Read 9 Done Sent Done false
16137 4994 Acked Write 6 Done Sent Done false
16138 4995 Acked Read 3 Done Sent Done false
16139 4996 Acked Write 9 Done Sent Done false
16140 4997 Acked Write 4 Done Sent Done false
16141 4998 Acked Read 6 Done Sent Done false
16142 4999 Acked Write 6 Done Sent Done false
16143 5000 Acked Flush 0 Done Sent Done false
16144 5001 Acked Read 9 Done Sent Done false
16145 5002 Acked Read 9 Done Sent Done false
16146 5003 Acked Read 6 Done Sent Done false
16147 5004 Acked Write 3 Done Sent Done false
16148 5005 Acked Read 5 Done Sent Done false
16149 5006 Acked Read 1 Done Sent Done false
16150 5007 Acked Write 2 Done Sent Done false
16151 5008 Acked Write 7 Done Sent Done false
16152 5009 Acked Read 1 Done Sent Done false
16153 5010 Acked Read 6 Done Sent Done false
16154 5011 Acked Write 5 Done Sent Done false
16155 5012 Acked Read 7 Done Sent Done false
16156 5013 Acked Read 10 Done Sent Done false
16157 5014 Acked Read 2 Done Sent Done false
16158 5015 Acked Flush 0 Done Sent Done false
16159 5016 Acked Flush 0 Done Sent Done false
16160 5017 Acked Read 5 Done Sent Done false
16161 5018 Acked Read 8 Done Sent Done false
16162 5019 Acked Read 8 Done Sent Done false
16163 5020 Acked Write 2 Done Sent Done false
16164 5021 Acked Read 3 Done Sent Done false
16165 5022 Acked Write 8 Done Sent Done false
16166 5023 Acked Write 3 Done Sent Done false
16167 5024 Acked Read 10 Done Sent Done false
16168 5025 Acked Read 10 Done Sent Done false
16169 5026 Acked Read 3 Done Sent Done false
16170 5027 Acked Read 9 Done Sent Done false
16171 5028 Acked Write 1 Done Sent Done false
16172 5029 Acked Write 7 Done Sent Done false
16173 5030 Acked Write 10 Done Sent Done false
16174 5031 Acked Write 3 Done Sent Done false
16175 5032 Acked Read 1 Done Sent Done false
16176 5033 Acked Write 5 Done Sent Done false
16177 5034 Acked Write 6 Done Sent Done false
16178 5035 Acked Write 5 Done Sent Done false
16179 5036 Acked Write 6 Done Sent Done false
16180 5037 Acked Read 10 Done Sent Done false
16181 5038 Acked Write 4 Done Sent Done false
16182 5039 Acked Read 5 Done Sent Done false
16183 5040 Acked Read 8 Done Sent Done false
16184 5041 Acked Read 6 Done Sent Done false
16185 5042 Acked Write 1 Done Sent Done false
16186 5043 Acked Read 6 Done Sent Done false
16187 5044 Acked Read 8 Done Sent Done false
16188 5045 Acked Read 1 Done Sent Done false
16189 5046 Acked Write 6 Done Sent Done false
16190 5047 Acked Write 3 Done Sent Done false
16191 5048 Acked Read 9 Done Sent Done false
16192 5049 Acked Write 2 Done Sent Done false
16193 5050 Acked Write 3 Done Sent Done false
16194 5051 Acked Write 7 Done Sent Done false
16195 5052 Acked Read 9 Done Sent Done false
16196 5053 Acked Write 1 Done Sent Done false
16197 5054 Acked Write 2 Done Sent Done false
16198 5055 Acked Read 7 Done Sent Done false
16199 5056 Acked Read 9 Done Sent Done false
16200 5057 Acked Write 2 Done Sent Done false
16201 5058 Acked Read 9 Done Sent Done false
16202 5059 Acked Write 5 Done Sent Done false
16203 5060 Acked Read 5 Done Sent Done false
16204 5061 Acked Flush 0 Done Sent Done false
16205 5062 Acked Write 8 Done Sent Done false
16206 5063 Acked Write 7 Done Sent Done false
16207 5064 Acked Write 7 Done Sent Done false
16208 5065 Acked Read 3 Done Sent Done false
16209 5066 Acked Read 6 Done Sent Done false
16210 5067 Acked Write 10 Done Sent Done false
16211 5068 Acked Read 5 Done Sent Done false
16212 5069 Acked Read 8 Done Sent Done false
16213 5070 Acked Write 1 Done Sent Done false
16214 5071 Acked Read 5 Done Sent Done false
16215 5072 Acked Read 4 Done Sent Done false
16216 5073 Acked Write 9 Done Sent Done false
16217 5074 Acked Read 10 Done Sent Done false
16218 5075 Acked Read 8 Done Sent Done false
16219 5076 Acked Write 4 Done Sent Done false
16220 5077 Acked Read 9 Done Sent Done false
16221 5078 Acked Read 9 Done Sent Done false
16222 5079 Acked Read 2 Done Sent Done false
16223 5080 Acked Read 6 Done Sent Done false
16224 5081 Acked Write 5 Done Sent Done false
16225 5082 Acked Write 4 Done Sent Done false
16226 5083 Acked Read 2 Done Sent Done false
16227 5084 Acked Write 9 Done Sent Done false
16228 5085 Acked Write 8 Done Sent Done false
16229 5086 Acked Write 1 Done Sent Done false
16230 5087 Acked Read 7 Done Sent Done false
16231 5088 Acked Read 8 Done Sent Done false
16232 5089 Acked Write 5 Done Sent Done false
16233 5090 Acked Flush 0 Done Sent Done false
16234 5091 Acked Read 9 Done Sent Done false
16235 5092 Acked Write 8 Done Sent Done false
16236 5093 Acked Write 8 Done Sent Done false
16237 5094 Acked Write 3 Done Sent Done false
16238 5095 Acked Read 10 Done Sent Done false
16239 5096 Acked Read 1 Done Sent Done false
16240 5097 Acked Read 6 Done Sent Done false
16241 5098 Acked Read 4 Done Sent Done false
16242 5099 Acked Flush 0 Done Sent Done false
16243 5100 Acked Write 6 Done Sent Done false
16244 5101 Acked Flush 0 Done Sent Done false
16245 5102 Acked Write 2 Done Sent Done false
16246 5103 Acked Read 2 Done Sent Done false
16247 5104 Acked Read 6 Done Sent Done false
16248 5105 Acked Write 7 Done Sent Done false
16249 5106 Acked Write 1 Done Sent Done false
16250 5107 Acked Flush 0 Done Sent Done false
16251 5108 Acked Write 5 Done Sent Done false
16252 5109 Acked Read 7 Done Sent Done false
16253 5110 Acked Write 7 Done Sent Done false
16254 5111 Acked Flush 0 Done Sent Done false
16255 5112 Acked Read 7 Done Sent Done false
16256 5113 Acked Write 10 Done Sent Done false
16257 5114 Acked Read 7 Done Sent Done false
16258 5115 Acked Read 6 Done Sent Done false
16259 5116 Acked Read 6 Done Sent Done false
16260 5117 Acked Write 7 Done Sent Done false
16261 5118 Acked Write 7 Done Sent Done false
16262 5119 Acked Read 2 Done Sent Done false
16263 5120 Acked Write 9 Done Sent Done false
16264 5121 Acked Read 3 Done Sent Done false
16265 5122 Acked Write 1 Done Sent Done false
16266 5123 Acked Write 8 Done Sent Done false
16267 5124 Acked Read 1 Done Sent Done false
16268 5125 Acked Read 9 Done Sent Done false
16269 5126 Acked Write 10 Done Sent Done false
16270 5127 Acked Read 4 Done Sent Done false
16271 5128 Acked Read 2 Done Sent Done false
16272 5129 Acked Read 8 Done Sent Done false
16273 5130 Acked Write 3 Done Sent Done false
16274 5131 Acked Read 6 Done Sent Done false
16275 5132 Acked Read 4 Done Sent Done false
16276 5133 Acked Read 2 Done Sent Done false
16277 5134 Acked Write 7 Done Sent Done false
16278 5135 Acked Read 8 Done Sent Done false
16279 5136 Acked Flush 0 Done Sent Done false
16280 5137 Acked Read 9 Done Sent Done false
16281 5138 Acked Read 10 Done Sent Done false
16282 5139 Acked Read 2 Done Sent Done false
16283 5140 Acked Read 4 Done Sent Done false
16284 5141 Acked Read 10 Done Sent Done false
16285 5142 Acked Read 5 Done Sent Done false
16286 5143 Acked Write 9 Done Sent Done false
16287 5144 Acked Flush 0 Done Sent Done false
16288 5145 Acked Write 8 Done Sent Done false
16289 5146 Acked Read 4 Done Sent Done false
16290 5147 Acked Write 9 Done Sent Done false
16291 5148 Acked Flush 0 Done Sent Done false
16292 5149 Acked Flush 0 Done Sent Done false
16293 5150 Acked Write 9 Done Sent Done false
16294 5151 Acked Write 7 Done Sent Done false
16295 5152 Acked Write 6 Done Sent Done false
16296 5153 Acked Write 1 Done Sent Done false
16297 5154 Acked Read 1 Done Sent Done false
16298 5155 Acked Write 10 Done Sent Done false
16299 5156 Acked Read 1 Done Sent Done false
16300 5157 Acked Write 3 Done Sent Done false
16301 5158 Acked Write 10 Done Sent Done false
16302 5159 Acked Read 3 Done Sent Done false
16303 5160 Acked Read 7 Done Sent Done false
16304 5161 Acked Read 2 Done Sent Done false
16305 5162 Acked Write 2 Done Sent Done false
16306 5163 Acked Read 10 Done Sent Done false
16307 5164 Acked Read 9 Done Sent Done false
16308 5165 Acked Read 6 Done Sent Done false
16309 5166 Acked Read 2 Done Sent Done false
16310 5167 Acked Write 4 Done Sent Done false
16311 5168 Acked Write 5 Done Sent Done false
16312 5169 Acked Write 5 Done Sent Done false
16313 5170 Acked Write 10 Done Sent Done false
16314 5171 Acked Write 6 Done Sent Done false
16315 5172 Acked Read 1 Done Sent Done false
16316 5173 Acked Read 5 Done Sent Done false
16317 5174 Acked Read 5 Done Sent Done false
16318 5175 Acked Write 9 Done Sent Done false
16319 5176 Acked Write 6 Done Sent Done false
16320 5177 Acked Read 10 Done Sent Done false
16321 5178 Acked Read 9 Done Sent Done false
16322 5179 Acked Write 4 Done Sent Done false
16323 5180 Acked Read 5 Done Sent Done false
16324 5181 Acked Write 8 Done Sent Done false
16325 5182 Acked Write 1 Done Sent Done false
16326 5183 Acked Read 2 Done Sent Done false
16327 5184 Acked Flush 0 Done Sent Done false
16328 5185 Acked Read 6 Done Sent Done false
16329 5186 Acked Read 1 Done Sent Done false
16330 5187 Acked Read 4 Done Sent Done false
16331 5188 Acked Write 7 Done Sent Done false
16332 5189 Acked Read 7 Done Sent Done false
16333 5190 Acked Write 9 Done Sent Done false
16334 5191 Acked Write 3 Done Sent Done false
16335 5192 Acked Write 7 Done Sent Done false
16336 5193 Acked Flush 0 Done Sent Done false
16337 5194 Acked Read 1 Done Sent Done false
16338 5195 Acked Write 8 Done Sent Done false
16339 5196 Acked Read 9 Done Sent Done false
16340 5197 Acked Read 6 Done Sent Done false
16341 5198 Acked Read 10 Done Sent Done false
16342 5199 Acked Write 10 Done Sent Done false
16343 5200 Acked Read 5 Done Sent Done false
16344 5201 Acked Read 4 Done Sent Done false
16345 5202 Acked Read 6 Done Sent Done false
16346 5203 Acked Write 10 Done Sent Done false
16347 5204 Acked Read 3 Done Sent Done false
16348 5205 Acked Write 10 Done Sent Done false
16349 5206 Acked Read 6 Done Sent Done false
16350 5207 Acked Write 5 Done Sent Done false
16351 5208 Acked Read 5 Done Sent Done false
16352 5209 Acked Flush 0 Done Sent Done false
16353 5210 Acked Write 9 Done Sent Done false
16354 5211 Acked Write 3 Done Sent Done false
16355 5212 Acked Read 5 Done Sent Done false
16356 5213 Acked Write 9 Done Sent Done false
16357 5214 Acked Read 2 Done Sent Done false
16358 5215 Acked Flush 0 Done Sent Done false
16359 5216 Acked Write 7 Done Sent Done false
16360 5217 Acked Read 3 Done Sent Done false
16361 5218 Acked Read 9 Done Sent Done false
16362 5219 Acked Read 3 Done Sent Done false
16363 5220 Acked Flush 0 Done Sent Done false
16364 5221 Acked Write 6 Done Sent Done false
16365 5222 Acked Read 1 Done Sent Done false
16366 5223 Acked Read 8 Done Sent Done false
16367 5224 Acked Read 3 Done Sent Done false
16368 5225 Acked Write 7 Done Sent Done false
16369 5226 Acked Write 1 Done Sent Done false
16370 5227 Acked Write 6 Done Sent Done false
16371 5228 Acked Read 7 Done Sent Done false
16372 5229 Acked Read 2 Done Sent Done false
16373 5230 Acked Read 2 Done Sent Done false
16374 5231 Acked Write 3 Done Sent Done false
16375 5232 Acked Write 9 Done Sent Done false
16376 5233 Acked Flush 0 Done Sent Done false
16377 STATES DS:0 DS:1 DS:2 TOTAL
16378 Sent 0 302 0 302
16379 Done 302 0 302 604
16380 Skipped 0 0 0 0
16381 Error 0 0 0 0
16382 Last Flush: 5233 4931 5233
16383 Downstairs last five completed: 4931 4930 4929 4928 4927
16384 Upstairs last five completed: 5233 5232 5231 5230 5229
16385 CLIENT: Up:0 ds:302 act:2
163862024-11-07T14:06:43.665ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
163872024-11-07T14:06:43.665ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50184) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
163882024-11-07T14:06:43.666ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
163892024-11-07T14:06:43.666ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
163902024-11-07T14:06:43.666ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
163912024-11-07T14:06:43.666ZINFOcrucible: send last flush ID to this DS: 4931 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
163922024-11-07T14:06:43.667ZINFOcrucible: Replied this last flush ID: 4931 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
163932024-11-07T14:06:43.667ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
163942024-11-07T14:06:43.667ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
163952024-11-07T14:06:43.667ZINFOcrucible: [1] client re-new 302 jobs since flush 4931 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
163962024-11-07T14:06:43.667ZINFOcrucible: [1] Replayed 302 jobs since flush: 4931 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
16397 ----------------------------------------------------------------
16398 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16399 Downstairs last five completed: 5233 5232 5231 5230 5229
16400 Upstairs last five completed: 5233 5232 5231 5230 5229
16401 CLIENT: Up:0 ds:0 act:3
16402 Replay: All jobs finished
16403 Main task has work to do, go find it
16404 got dsc Stop(0)
16405 stop 0
16406 Nov 07 14:06:45.960[8810] Got stop action so:false kr:false
16407 INFO request completed, latency_us: 353, response_code: 204, uri: /stop/cid/0, method: POST, req_id: cf41b619-f555-41b7-86a2-4232129597b2, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
164082024-11-07T14:06:45.961ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
16409 Nov 07 14:06:45.961 INFO request completed, latency_us: 132, response_code: 200, uri: /state/cid/0, method: GET, req_id{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-07T14:06:45.961664229Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"":"io task","client":"0","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
16410 : b85b09f7-f649-40e0-adde-8f530cc297dd, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-07T14:06:45.961696137Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
16411 
164122024-11-07T14:06:45.961ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
164132024-11-07T14:06:45.961ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
164142024-11-07T14:06:45.961ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
16415 [8810] Exited with: ExitStatus(unix_wait_status(9))
16416 [8810][0] reports Exit
16417 Nov 07 14:06:49.963 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/0, method: GET, req_id: 89d99472-9821-496a-99ae-6e3f77355467, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
16418 001/300 Read block 4492 len 20480
16419 002/300 Write block 2882 len 32768 data: 5 5 5 5 4 4 4 3
16420 003/300 Read block 139 len 40960
16421 004/300 Read block 3137 len 4096
16422 005/300 Write block 2219 len 8192 data: 2 2
16423 006/300 Flush
16424 007/300 Write block 1614 len 32768 data: 2 2 2 2 2 2 2 2
16425 008/300 Read block 291 len 4096
16426 009/300 Write block 3102 len 32768 data: 2 2 2 4 3 3 3 2
16427 010/300 Read block 180 len 8192
16428 011/300 Read block 4 len 24576
16429 012/300 Read block 1498 len 32768
16430 013/300 Read block 3587 len 4096
16431 014/300 Read block 3030 len 32768
16432 015/300 Read block 323 len 32768
16433 016/300 Read block 4141 len 4096
16434 017/300 Flush
16435 018/300 Read block 2072 len 28672
16436 019/300 Read block 434 len 40960
16437 020/300 Read block 3528 len 16384
16438 021/300 Read block 2139 len 12288
16439 022/300 Flush
16440 023/300 Write block 1008 len 40960 data: 3 3 3 2 3 3 4 5 4 4
16441 024/300 Write block 2924 len 20480 data: 2 2 2 2 2
16442 025/300 Read block 1481 len 36864
16443 026/300 Write block 3719 len 28672 data: 6 6 5 5 4 4 4
16444 027/300 Flush
16445 028/300 Read block 2341 len 24576
16446 029/300 Read block 701 len 40960
16447 030/300 Write block 3227 len 16384 data: 2 2 2 3
16448 031/300 Read block 3120 len 12288
16449 032/300 Read block 1635 len 36864
16450 033/300 Write block 4701 len 20480 data: 5 5 3 2 2
16451 034/300 Read block 3625 len 28672
16452 035/300 Read block 4527 len 36864
16453 036/300 Flush
16454 037/300 Write block 3716 len 36864 data: 5 6 7 7 7 6 6 5 5
16455 038/300 Read block 993 len 40960
16456 039/300 Read block 1856 len 8192
16457 040/300 Read block 4984 len 24576
16458 041/300 Read block 774 len 28672
16459 042/300 Read block 174 len 20480
16460 043/300 Read block 3022 len 8192
16461 044/300 Read block 2449 len 24576
16462 045/300 Read block 1258 len 8192
16463 046/300 Write block 1490 len 20480 data: 2 2 2 3 3
16464 047/300 Write block 367 len 36864 data: 3 4 3 3 3 3 3 2 2
16465 048/300 Write block 2647 len 24576 data: 3 4 4 4 4 2
16466 049/300 Read block 1615 len 20480
16467 050/300 Write block 4966 len 36864 data: 4 5 5 4 4 3 2 3 3
16468 051/300 Write block 2327 len 28672 data: 3 3 3 3 3 3 3
16469 052/300 Read block 2907 len 8192
16470 053/300 Write block 227 len 8192 data: 6 2
16471 054/300 Read block 1180 len 24576
16472 055/300 Read block 14 len 12288
16473 056/300 Read block 483 len 28672
16474 057/300 Write block 4271 len 32768 data: 7 7 6 5 5 4 5 4
16475 058/300 Write block 2414 len 28672 data: 2 2 2 2 3 4 4
16476 059/300 Write block 4228 len 32768 data: 6 6 5 4 4 3 3 3
16477 060/300 Read block 4688 len 36864
16478 061/300 Write block 4065 len 20480 data: 3 3 3 3 3
16479 062/300 Read block 2073 len 36864
16480 063/300 Read block 2837 len 20480
16481 064/300 Flush
16482 065/300 Write block 1365 len 20480 data: 4 4 3 3 3
16483 066/300 Read block 2145 len 16384
16484 067/300 Write block 3392 len 28672 data: 2 3 5 5 4 3 3
16485 068/300 Write block 4577 len 24576 data: 4 4 4 3 3 2
16486 069/300 Read block 2386 len 36864
16487 070/300 Read block 4915 len 32768
16488 071/300 Write block 4300 len 32768 data: 5 5 4 4 2 2 3 3
16489 072/300 Write block 186 len 12288 data: 5 4 4
16490 073/300 Read block 2973 len 4096
16491 074/300 Read block 2108 len 32768
16492 075/300 Read block 429 len 8192
16493 076/300 Write block 4017 len 24576 data: 4 4 3 4 5 5
16494 077/300 Write block 1708 len 20480 data: 3 3 3 3 2
16495 078/300 Read block 3631 len 24576
16496 079/300 Write block 3478 len 4096 data: 6
16497 080/300 Read block 2532 len 8192
16498 081/300 Read block 845 len 16384
16499 082/300 Write block 4841 len 32768 data: 4 6 6 6 5 5 5 3
16500 083/300 Write block 1791 len 4096 data: 7
16501 084/300 Write block 26 len 12288 data: 6 6 6
16502 085/300 Read block 2641 len 32768
16503 086/300 Write block 2615 len 12288 data: 4 4 5
16504 087/300 Flush
16505 088/300 Read block 670 len 28672
16506 089/300 Write block 1579 len 8192 data: 3 4
16507 090/300 Write block 1735 len 36864 data: 2 2 2 3 3 2 3 4 4
16508 091/300 Write block 3661 len 8192 data: 5 4
16509 092/300 Read block 2342 len 36864
16510 093/300 Write block 3377 len 40960 data: 7 6 5 5 5 4 4 4 4 3
16511 094/300 Read block 3094 len 16384
16512 095/300 Read block 1564 len 4096
16513 096/300 Read block 2844 len 36864
16514 097/300 Read block 4804 len 4096
16515 098/300 Write block 2710 len 8192 data: 3 3
16516 099/300 Read block 3836 len 16384
16517 100/300 Write block 1919 len 40960 data: 4 4 4 4 2 3 4 4 5 5
16518 101/300 Read block 263 len 24576
16519 102/300 Write block 3694 len 12288 data: 6 6 5
16520 103/300 Read block 1098 len 28672
16521 104/300 Read block 3593 len 28672
16522 105/300 Flush
16523 106/300 Read block 3573 len 36864
16524 107/300 Read block 1794 len 28672
16525 108/300 Read block 2188 len 40960
16526 109/300 Write block 4122 len 28672 data: 6 6 6 5 5 5 3
16527 110/300 Flush
16528 111/300 Flush
16529 112/300 Write block 3626 len 32768 data: 5 5 6 7 8 6 6 4
16530 113/300 Read block 3915 len 4096
16531 114/300 Read block 4974 len 24576
16532 115/300 Read block 4078 len 8192
16533 116/300 Write block 4033 len 4096 data: 2
16534 117/300 Write block 3441 len 16384 data: 5 5 5 4
16535 118/300 Write block 1934 len 20480 data: 2 2 2 2 2
16536 119/300 Read block 2148 len 32768
16537 120/300 Read block 3223 len 4096
16538 121/300 Write block 466 len 8192 data: 4 4
16539 122/300 Read block 2796 len 40960
16540 123/300 Write block 406 len 24576 data: 5 4 4 4 5 5
16541 124/300 Read block 1191 len 16384
16542 125/300 Read block 4669 len 28672
16543 126/300 Write block 4661 len 36864 data: 2 2 2 2 2 2 2 3 4
16544 127/300 Flush
16545 128/300 Write block 452 len 28672 data: 8 6 7 5 6 6 4
16546 129/300 Write block 4845 len 36864 data: 6 6 6 4 3 3 2 2 2
16547 130/300 Write block 297 len 32768 data: 3 3 3 3 5 3 3 4
16548 131/300 Read block 4535 len 12288
16549 132/300 Read block 971 len 8192
16550 133/300 Write block 3237 len 36864 data: 5 5 4 7 6 6 5 5 5
16551 134/300 Write block 3118 len 8192 data: 4 4
16552 135/300 Read block 4840 len 4096
16553 136/300 Read block 24 len 40960
16554 137/300 Read block 3021 len 28672
16555 138/300 Read block 3576 len 36864
16556 139/300 Read block 1152 len 28672
16557 140/300 Write block 3430 len 8192 data: 5 5
16558 141/300 Read block 2418 len 12288
16559 142/300 Write block 3862 len 36864 data: 6 6 6 5 4 4 4 5 6
16560 143/300 Write block 4913 len 32768 data: 3 3 5 5 5 6 7 7
16561 144/300 Write block 905 len 36864 data: 5 5 4 4 4 3 3 3 4
16562 145/300 Write block 3308 len 36864 data: 4 4 4 4 4 3 2 2 2
16563 146/300 Read block 375 len 16384
16564 147/300 Read block 4975 len 20480
16565 148/300 Read block 3938 len 12288
16566 149/300 Write block 2320 len 40960 data: 2 3 3 3 3 2 2 4 4 4
16567 150/300 Read block 734 len 20480
16568 151/300 Flush
16569 152/300 Write block 1940 len 28672 data: 3 3 3 5 6 6 6
16570 153/300 Write block 2541 len 20480 data: 10 9 9 8 7
16571 154/300 Read block 2912 len 8192
16572 155/300 Read block 2972 len 20480
16573 156/300 Write block 1380 len 36864 data: 2 2 2 2 2 3 3 3 3
16574 157/300 Write block 2036 len 40960 data: 2 2 2 2 3 3 3 4 4 4
16575 158/300 Read block 3020 len 40960
16576 159/300 Read block 3722 len 40960
16577 160/300 Write block 3323 len 16384 data: 5 5 5 5
16578 161/300 Read block 2111 len 32768
16579 162/300 Write block 4118 len 20480 data: 4 6 5 6 7
16580 163/300 Read block 2002 len 24576
16581 164/300 Read block 681 len 32768
16582 165/300 Write block 3633 len 16384 data: 5 4 4 3
16583 166/300 Read block 1138 len 8192
16584 167/300 Read block 2309 len 8192
16585 168/300 Write block 1507 len 4096 data: 4
16586 169/300 Read block 1543 len 24576
16587 170/300 Read block 4458 len 20480
16588 171/300 Write block 702 len 12288 data: 2 3 5
16589 172/300 Read block 115 len 32768
16590 173/300 Write block 3 len 24576 data: 4 4 3 3 2 2
16591 174/300 Read block 4541 len 4096
16592 175/300 Read block 4823 len 16384
16593 176/300 Flush
16594 177/300 Write block 4759 len 4096 data: 5
16595 178/300 Write block 3331 len 8192 data: 5 5
16596 179/300 Write block 834 len 12288 data: 4 3 3
16597 180/300 Write block 3707 len 36864 data: 4 4 3 4 5 4 3 4 5
16598 181/300 Write block 3404 len 32768 data: 6 4 4 2 2 2 2 3
16599 182/300 Read block 3170 len 40960
16600 183/300 Write block 1317 len 4096 data: 5
16601 184/300 Read block 536 len 4096
16602 185/300 Read block 2633 len 16384
16603 186/300 Read block 3948 len 32768
16604 187/300 Read block 763 len 12288
16605 188/300 Write block 2336 len 16384 data: 3 3 3 3
16606 189/300 Read block 1418 len 20480
16607 190/300 Write block 723 len 8192 data: 5 5
16608 191/300 Write block 1092 len 20480 data: 3 3 3 3 3
16609 192/300 Write block 4538 len 32768 data: 5 4 3 4 4 4 4 4
16610 193/300 Read block 2021 len 8192
16611 194/300 Flush
16612 195/300 Read block 4642 len 28672
16613 196/300 Write block 3690 len 36864 data: 2 3 5 6 7 7 6 5 6
16614 197/300 Flush
16615 198/300 Read block 2962 len 24576
16616 199/300 Write block 2167 len 24576 data: 6 6 6 5 4 3
16617 200/300 Read block 2157 len 24576
16618 201/300 Write block 2611 len 40960 data: 4 5 4 5 5 5 6 5 4 5
16619 202/300 Write block 1958 len 24576 data: 3 3 4 4 3 3
16620 203/300 Read block 4451 len 12288
16621 204/300 Read block 4377 len 16384
16622 205/300 Read block 533 len 8192
16623 206/300 Read block 2200 len 32768
16624 207/300 Write block 3828 len 28672 data: 6 5 4 4 3 4 4
16625 208/300 Read block 784 len 12288
16626 209/300 Write block 223 len 24576 data: 2 3 3 6 7 3
16627 210/300 Write block 1010 len 24576 data: 4 3 4 4 5 6
16628 211/300 Read block 4920 len 28672
16629 212/300 Write block 1624 len 12288 data: 2 2 2
16630 213/300 Read block 3660 len 24576
16631 214/300 Write block 1478 len 16384 data: 3 3 3 3
16632 215/300 Read block 2738 len 24576
16633 216/300 Write block 376 len 24576 data: 3 3 3 5 7 8
16634 217/300 Write block 4188 len 4096 data: 3
16635 218/300 Read block 2448 len 40960
16636 219/300 Flush
16637 220/300 Write block 2650 len 32768 data: 5 5 3 2 2 2 3 3
16638 221/300 Read block 2537 len 12288
16639 222/300 Write block 4723 len 12288 data: 3 4 4
16640 223/300 Read block 2476 len 12288
16641 224/300 Read block 143 len 36864
16642 225/300 Write block 4880 len 32768 data: 2 2 3 3 3 3 2 2
16643 226/300 Write block 1563 len 20480 data: 5 6 5 5 5
16644 227/300 Read block 4890 len 16384
16645 228/300 Read block 2065 len 36864
16646 229/300 Read block 4586 len 20480
16647 230/300 Read block 4424 len 16384
16648 231/300 Read block 263 len 36864
16649 232/300 Read block 3929 len 16384
16650 233/300 Read block 4063 len 24576
16651 234/300 Read block 4812 len 24576
16652 235/300 Write block 2732 len 12288 data: 6 5 4
16653 236/300 Write block 3220 len 20480 data: 4 3 2 2 2
16654 237/300 Read block 678 len 24576
16655 238/300 Write block 239 len 4096 data: 5
16656 239/300 Write block 2821 len 28672 data: 4 4 4 5 5 6 7
16657 240/300 Write block 3579 len 36864 data: 2 3 4 4 5 5 5 5 4
16658 241/300 Write block 2112 len 20480 data: 2 2 2 2 2
16659 242/300 Read block 3215 len 24576
16660 243/300 Read block 97 len 36864
16661 244/300 Write block 2551 len 40960 data: 4 3 2 4 4 4 4 3 3 3
16662 245/300 Read block 4633 len 8192
16663 246/300 Flush
16664 247/300 Write block 1861 len 12288 data: 4 3 3
16665 248/300 Read block 1484 len 20480
16666 249/300 Write block 4751 len 12288 data: 4 3 4
16667 250/300 Read block 2006 len 16384
16668 251/300 Write block 900 len 20480 data: 8 7 6 8 7
16669 252/300 Read block 1296 len 12288
16670 253/300 Write block 1934 len 4096 data: 3
16671 254/300 Write block 4602 len 4096 data: 5
16672 255/300 Write block 69 len 16384 data: 2 2 2 5
16673 256/300 Read block 1390 len 4096
16674 257/300 Read block 1012 len 32768
16675 258/300 Write block 2222 len 20480 data: 2 2 2 2 4
16676 259/300 Write block 4905 len 40960 data: 2 3 3 4 4 3 4 4 4 4
16677 260/300 Read block 654 len 40960
16678 261/300 Read block 3557 len 24576
16679 262/300 Write block 3965 len 36864 data: 5 4 4 4 4 3 3 5 5
16680 263/300 Write block 2962 len 32768 data: 3 3 4 3 2 2 2 4
16681 264/300 Write block 508 len 20480 data: 2 2 3 3 4
16682 265/300 Flush
16683 266/300 Read block 4639 len 28672
16684 267/300 Read block 231 len 12288
16685 268/300 Read block 4167 len 40960
16686 269/300 Flush
16687 270/300 Flush
16688 271/300 Read block 926 len 4096
16689 272/300 Read block 4666 len 8192
16690 273/300 Write block 331 len 20480 data: 4 4 5 4 5
16691 274/300 Read block 2300 len 20480
16692 275/300 Read block 3973 len 32768
16693 276/300 Write block 1948 len 20480 data: 5 5 5 3 2
16694 277/300 Flush
16695 278/300 Flush
16696 279/300 Read block 4797 len 8192
16697 280/300 Flush
16698 281/300 Write block 3295 len 16384 data: 5 5 5 5
16699 282/300 Read block 2398 len 28672
16700 283/300 Flush
16701 284/300 Write block 1327 len 28672 data: 4 4 5 5 4 5 6
16702 285/300 Write block 912 len 32768 data: 4 5 5 5 5 5 5 6
16703 286/300 Read block 1416 len 28672
16704 287/300 Read block 690 len 36864
16705 288/300 Read block 780 len 40960
16706 289/300 Write block 3577 len 8192 data: 3 2
16707 290/300 Read block 2979 len 12288
16708 291/300 Flush
16709 292/300 Write block 2809 len 36864 data: 4 3 2 2 2 3 3 4 5
16710 293/300 Write block 3990 len 16384 data: 3 5 4 4
16711 294/300 Read block 6 len 32768
16712 295/300 Write block 3672 len 16384 data: 2 2 3 3
16713 296/300 Write block 259 len 20480 data: 3 3 3 4 4
16714 297/300 Read block 3910 len 36864
16715 298/300 Write block 3028 len 20480 data: 5 5 4 4 4
16716 299/300 Write block 4184 len 4096 data: 2
16717 300/300 Flush
16718 301/300 Flush
16719 Main task has work to do, go find it
16720 got dsc Start(0)
16721 start 0
16722 Nov 07 14:06:50.252 INFO request completed, latency_us: 211, response_code: 204, uri: /start/cid/0, method: POST, req_id: 9668ac59-dd7a-4f28-84c0-323c717304ca, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
16723 Replay: started 0, returned:Ok(())
16724 ----------------------------------------------------------------
16725 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16726 States: Offline Active Active
16727 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16728 5234 Acked Read 5 Sent Done Done false
16729 5235 Acked Write 8 Sent Done Done false
16730 5236 Acked Read 10 Sent Done Done false
16731 5237 Acked Read 1 Sent Done Done false
16732 5238 Acked Write 2 Sent Done Done false
16733 5239 Acked Flush 0 Sent Done Done false
16734 5240 Acked Write 8 Sent Done Done false
16735 5241 Acked Read 1 Sent Done Done false
16736 5242 Acked Write 8 Sent Done Done false
16737 5243 Acked Read 2 Sent Done Done false
16738 5244 Acked Read 6 Sent Done Done false
16739 5245 Acked Read 8 Sent Done Done false
16740 5246 Acked Read 1 Sent Done Done false
16741 5247 Acked Read 8 Sent Done Done false
16742 5248 Acked Read 8 Sent Done Done false
16743 5249 Acked Read 1 Sent Done Done false
16744 5250 Acked Flush 0 Sent Done Done false
16745 5251 Acked Read 7 Sent Done Done false
16746 5252 Acked Read 10 Sent Done Done false
16747 5253 Acked Read 4 Sent Done Done false
16748 5254 Acked Read 3 Sent Done Done false
16749 5255 Acked Flush 0 Sent Done Done false
16750 5256 Acked Write 10 Sent Done Done false
16751 5257 Acked Write 5 Sent Done Done false
16752 5258 Acked Read 9 Sent Done Done false
16753 5259 Acked Write 7 Sent Done Done false
16754 5260 Acked Flush 0 Sent Done Done false
16755 5261 Acked Read 6 Sent Done Done false
16756 5262 Acked Read 10 Sent Done Done false
16757 5263 Acked Write 4 Sent Done Done false
16758 5264 Acked Read 3 Sent Done Done false
16759 5265 Acked Read 9 Sent Done Done false
16760 5266 Acked Write 5 Sent Done Done false
16761 5267 Acked Read 7 Sent Done Done false
16762 5268 Acked Read 9 Sent Done Done false
16763 5269 Acked Flush 0 Sent Done Done false
16764 5270 Acked Write 9 Sent Done Done false
16765 5271 Acked Read 10 Sent Done Done false
16766 5272 Acked Read 2 Sent Done Done false
16767 5273 Acked Read 6 Sent Done Done false
16768 5274 Acked Read 7 Sent Done Done false
16769 5275 Acked Read 5 Sent Done Done false
16770 5276 Acked Read 2 Sent Done Done false
16771 5277 Acked Read 6 Sent Done Done false
16772 5278 Acked Read 2 Sent Done Done false
16773 5279 Acked Write 5 Sent Done Done false
16774 5280 Acked Write 9 Sent Done Done false
16775 5281 Acked Write 6 Sent Done Done false
16776 5282 Acked Read 5 Sent Done Done false
16777 5283 Acked Write 9 Sent Done Done false
16778 5284 Acked Write 7 Sent Done Done false
16779 5285 Acked Read 2 Sent Done Done false
16780 5286 Acked Write 2 Sent Done Done false
16781 5287 Acked Read 6 Sent Done Done false
16782 5288 Acked Read 3 Sent Done Done false
16783 5289 Acked Read 7 Sent Done Done false
16784 5290 Acked Write 8 Sent Done Done false
16785 5291 Acked Write 7 Sent Done Done false
16786 5292 Acked Write 8 Sent Done Done false
16787 5293 Acked Read 9 Sent Done Done false
16788 5294 Acked Write 5 Sent Done Done false
16789 5295 Acked Read 9 Sent Done Done false
16790 5296 Acked Read 5 Sent Done Done false
16791 5297 Acked Flush 0 Sent Done Done false
16792 5298 Acked Write 5 Sent Done Done false
16793 5299 Acked Read 4 Sent Done Done false
16794 5300 Acked Write 7 Sent Done Done false
16795 5301 Acked Write 6 Sent Done Done false
16796 5302 Acked Read 9 Sent Done Done false
16797 5303 Acked Read 8 Sent Done Done false
16798 5304 Acked Write 8 Sent Done Done false
16799 5305 Acked Write 3 Sent Done Done false
16800 5306 Acked Read 1 Sent Done Done false
16801 5307 Acked Read 8 Sent Done Done false
16802 5308 Acked Read 2 Sent Done Done false
16803 5309 Acked Write 6 Sent Done Done false
16804 5310 Acked Write 5 Sent Done Done false
16805 5311 Acked Read 6 Sent Done Done false
16806 5312 Acked Write 1 Sent Done Done false
16807 5313 Acked Read 2 Sent Done Done false
16808 5314 Acked Read 4 Sent Done Done false
16809 5315 Acked Write 8 Sent Done Done false
16810 5316 Acked Write 1 Sent Done Done false
16811 5317 Acked Write 3 Sent Done Done false
16812 5318 Acked Read 8 Sent Done Done false
16813 5319 Acked Write 3 Sent Done Done false
16814 5320 Acked Flush 0 Sent Done Done false
16815 5321 Acked Read 7 Sent Done Done false
16816 5322 Acked Write 2 Sent Done Done false
16817 5323 Acked Write 9 Sent Done Done false
16818 5324 Acked Write 2 Sent Done Done false
16819 5325 Acked Read 9 Sent Done Done false
16820 5326 Acked Write 10 Sent Done Done false
16821 5327 Acked Read 4 Sent Done Done false
16822 5328 Acked Read 1 Sent Done Done false
16823 5329 Acked Read 9 Sent Done Done false
16824 5330 Acked Read 1 Sent Done Done false
16825 5331 Acked Write 2 Sent Done Done false
16826 5332 Acked Read 4 Sent Done Done false
16827 5333 Acked Write 10 Sent Done Done false
16828 5334 Acked Read 6 Sent Done Done false
16829 5335 Acked Write 3 Sent Done Done false
16830 5336 Acked Read 7 Sent Done Done false
16831 5337 Acked Read 7 Sent Done Done false
16832 5338 Acked Flush 0 Sent Done Done false
16833 5339 Acked Read 9 Sent Done Done false
16834 5340 Acked Read 7 Sent Done Done false
16835 5341 Acked Read 10 Sent Done Done false
16836 5342 Acked Write 7 Sent Done Done false
16837 5343 Acked Flush 0 Sent Done Done false
16838 5344 Acked Flush 0 Sent Done Done false
16839 5345 Acked Write 8 Sent Done Done false
16840 5346 Acked Read 1 Sent Done Done false
16841 5347 Acked Read 6 Sent Done Done false
16842 5348 Acked Read 2 Sent Done Done false
16843 5349 Acked Write 1 Sent Done Done false
16844 5350 Acked Write 4 Sent Done Done false
16845 5351 Acked Write 5 Sent Done Done false
16846 5352 Acked Read 8 Sent Done Done false
16847 5353 Acked Read 1 Sent Done Done false
16848 5354 Acked Write 2 Sent Done Done false
16849 5355 Acked Read 10 Sent Done Done false
16850 5356 Acked Write 6 Sent Done Done false
16851 5357 Acked Read 4 Sent Done Done false
16852 5358 Acked Read 7 Sent Done Done false
16853 5359 Acked Write 9 Sent Done Done false
16854 5360 Acked Flush 0 Sent Done Done false
16855 5361 Acked Write 7 Sent Done Done false
16856 5362 Acked Write 9 Sent Done Done false
16857 5363 Acked Write 8 Sent Done Done false
16858 5364 Acked Read 3 Sent Done Done false
16859 5365 Acked Read 2 Sent Done Done false
16860 5366 Acked Write 9 Sent Done Done false
16861 5367 Acked Write 2 Sent Done Done false
16862 5368 Acked Read 1 Sent Done Done false
16863 5369 Acked Read 10 Sent Done Done false
16864 5370 Acked Read 7 Sent Done Done false
16865 5371 Acked Read 9 Sent Done Done false
16866 5372 Acked Read 7 Sent Done Done false
16867 5373 Acked Write 2 Sent Done Done false
16868 5374 Acked Read 3 Sent Done Done false
16869 5375 Acked Write 9 Sent Done Done false
16870 5376 Acked Write 8 Sent Done Done false
16871 5377 Acked Write 9 Sent Done Done false
16872 5378 Acked Write 9 Sent Done Done false
16873 5379 Acked Read 4 Sent Done Done false
16874 5380 Acked Read 5 Sent Done Done false
16875 5381 Acked Read 3 Sent Done Done false
16876 5382 Acked Write 10 Sent Done Done false
16877 5383 Acked Read 5 Sent Done Done false
16878 5384 Acked Flush 0 Sent Done Done false
16879 5385 Acked Write 7 Sent Done Done false
16880 5386 Acked Write 5 Sent Done Done false
16881 5387 Acked Read 2 Sent Done Done false
16882 5388 Acked Read 5 Sent Done Done false
16883 5389 Acked Write 9 Sent Done Done false
16884 5390 Acked Write 10 Sent Done Done false
16885 5391 Acked Read 10 Sent Done Done false
16886 5392 Acked Read 10 Sent Done Done false
16887 5393 Acked Write 4 Sent Done Done false
16888 5394 Acked Read 8 Sent Done Done false
16889 5395 Acked Write 5 Sent Done Done false
16890 5396 Acked Read 6 Sent Done Done false
16891 5397 Acked Read 8 Sent Done Done false
16892 5398 Acked Write 4 Sent Done Done false
16893 5399 Acked Read 2 Sent Done Done false
16894 5400 Acked Read 2 Sent Done Done false
16895 5401 Acked Write 1 Sent Done Done false
16896 5402 Acked Read 6 Sent Done Done false
16897 5403 Acked Read 5 Sent Done Done false
16898 5404 Acked Write 3 Sent Done Done false
16899 5405 Acked Read 8 Sent Done Done false
16900 5406 Acked Write 6 Sent Done Done false
16901 5407 Acked Read 1 Sent Done Done false
16902 5408 Acked Read 4 Sent Done Done false
16903 5409 Acked Flush 0 Sent Done Done false
16904 5410 Acked Write 1 Sent Done Done false
16905 5411 Acked Write 2 Sent Done Done false
16906 5412 Acked Write 3 Sent Done Done false
16907 5413 Acked Write 9 Sent Done Done false
16908 5414 Acked Write 8 Sent Done Done false
16909 5415 Acked Read 10 Sent Done Done false
16910 5416 Acked Write 1 Sent Done Done false
16911 5417 Acked Read 1 Sent Done Done false
16912 5418 Acked Read 4 Sent Done Done false
16913 5419 Acked Read 8 Sent Done Done false
16914 5420 Acked Read 3 Sent Done Done false
16915 5421 Acked Write 4 Sent Done Done false
16916 5422 Acked Read 5 Sent Done Done false
16917 5423 Acked Write 2 Sent Done Done false
16918 5424 Acked Write 5 Sent Done Done false
16919 5425 Acked Write 8 Sent Done Done false
16920 5426 Acked Read 2 Sent Done Done false
16921 5427 Acked Flush 0 Sent Done Done false
16922 5428 Acked Read 7 Sent Done Done false
16923 5429 Acked Write 9 Sent Done Done false
16924 5430 Acked Flush 0 Sent Done Done false
16925 5431 Acked Read 6 Sent Done Done false
16926 5432 Acked Write 6 Sent Done Done false
16927 5433 Acked Read 6 Sent Done Done false
16928 5434 Acked Write 10 Sent Done Done false
16929 5435 Acked Write 6 Sent Done Done false
16930 5436 Acked Read 3 Sent Done Done false
16931 5437 Acked Read 4 Sent Done Done false
16932 5438 Acked Read 2 Sent Done Done false
16933 5439 Acked Read 8 Sent Done Done false
16934 5440 Acked Write 7 Sent Done Done false
16935 5441 Acked Read 3 Sent Done Done false
16936 5442 Acked Write 6 Sent Done Done false
16937 5443 Acked Write 6 Sent Done Done false
16938 5444 Acked Read 7 Sent Done Done false
16939 5445 Acked Flush 0 Sent Done Done false
16940 5446 Acked Write 3 Sent Done Done false
16941 5447 Acked Read 6 Sent Done Done false
16942 5448 Acked Write 4 Sent Done Done false
16943 5449 Acked Read 6 Sent Done Done false
16944 5450 Acked Write 6 Sent Done Done false
16945 5451 Acked Write 1 Sent Done Done false
16946 5452 Acked Read 10 Sent Done Done false
16947 5453 Acked Flush 0 Sent Done Done false
16948 5454 Acked Write 8 Sent Done Done false
16949 5455 Acked Read 3 Sent Done Done false
16950 5456 Acked Write 3 Sent Done Done false
16951 5457 Acked Read 3 Sent Done Done false
16952 5458 Acked Read 9 Sent Done Done false
16953 5459 Acked Write 8 Sent Done Done false
16954 5460 Acked Write 5 Sent Done Done false
16955 5461 Acked Read 4 Sent Done Done false
16956 5462 Acked Read 9 Sent Done Done false
16957 5463 Acked Read 5 Sent Done Done false
16958 5464 Acked Read 4 Sent Done Done false
16959 5465 Acked Read 9 Sent Done Done false
16960 5466 Acked Read 4 Sent Done Done false
16961 5467 Acked Read 6 Sent Done Done false
16962 5468 Acked Read 6 Sent Done Done false
16963 5469 Acked Write 3 Sent Done Done false
16964 5470 Acked Write 5 Sent Done Done false
16965 5471 Acked Read 6 Sent Done Done false
16966 5472 Acked Write 1 Sent Done Done false
16967 5473 Acked Write 7 Sent Done Done false
16968 5474 Acked Write 9 Sent Done Done false
16969 5475 Acked Write 5 Sent Done Done false
16970 5476 Acked Read 6 Sent Done Done false
16971 5477 Acked Read 9 Sent Done Done false
16972 5478 Acked Write 10 Sent Done Done false
16973 5479 Acked Read 2 Sent Done Done false
16974 5480 Acked Flush 0 Sent Done Done false
16975 5481 Acked Write 3 Sent Done Done false
16976 5482 Acked Read 5 Sent Done Done false
16977 5483 Acked Write 3 Sent Done Done false
16978 5484 Acked Read 4 Sent Done Done false
16979 5485 Acked Write 5 Sent Done Done false
16980 5486 Acked Read 3 Sent Done Done false
16981 5487 Acked Write 1 Sent Done Done false
16982 5488 Acked Write 1 Sent Done Done false
16983 5489 Acked Write 4 Sent Done Done false
16984 5490 Acked Read 1 Sent Done Done false
16985 5491 Acked Read 8 Sent Done Done false
16986 5492 Acked Write 5 Sent Done Done false
16987 5493 Acked Write 10 Sent Done Done false
16988 5494 Acked Read 10 Sent Done Done false
16989 5495 Acked Read 6 Sent Done Done false
16990 5496 Acked Write 9 Sent Done Done false
16991 5497 Acked Write 8 Sent Done Done false
16992 5498 Acked Write 5 Sent Done Done false
16993 5499 Acked Flush 0 Sent Done Done false
16994 5500 Acked Read 7 Sent Done Done false
16995 5501 Acked Read 3 Sent Done Done false
16996 5502 Acked Read 10 Sent Done Done false
16997 5503 Acked Flush 0 Sent Done Done false
16998 5504 Acked Flush 0 Sent Done Done false
16999 5505 Acked Read 1 Sent Done Done false
17000 5506 Acked Read 2 Sent Done Done false
17001 5507 Acked Write 5 Sent Done Done false
17002 5508 Acked Read 5 Sent Done Done false
17003 5509 Acked Read 8 Sent Done Done false
17004 5510 Acked Write 5 Sent Done Done false
17005 5511 Acked Flush 0 Sent Done Done false
17006 5512 Acked Flush 0 Sent Done Done false
17007 5513 Acked Read 2 Sent Done Done false
17008 5514 Acked Flush 0 Sent Done Done false
17009 5515 Acked Write 4 Sent Done Done false
17010 5516 Acked Read 7 Sent Done Done false
17011 5517 Acked Flush 0 Sent Done Done false
17012 5518 Acked Write 7 Sent Done Done false
17013 5519 Acked Write 8 Sent Done Done false
17014 5520 Acked Read 7 Sent Done Done false
17015 5521 Acked Read 9 Sent Done Done false
17016 5522 Acked Read 10 Sent Done Done false
17017 5523 Acked Write 2 Sent Done Done false
17018 5524 Acked Read 3 Sent Done Done false
17019 5525 Acked Flush 0 Sent Done Done false
17020 5526 Acked Write 9 Sent Done Done false
17021 5527 Acked Write 4 Sent Done Done false
17022 5528 Acked Read 8 Sent Done Done false
17023 5529 Acked Write 4 Sent Done Done false
17024 5530 Acked Write 5 Sent Done Done false
17025 5531 Acked Read 9 Sent Done Done false
17026 5532 Acked Write 5 Sent Done Done false
17027 5533 Acked Write 1 Sent Done Done false
17028 5534 Acked Flush 0 Sent Done Done false
17029 5535 Acked Flush 0 Sent Done Done false
17030 STATES DS:0 DS:1 DS:2 TOTAL
17031 Sent 302 0 0 302
17032 Done 0 302 302 604
17033 Skipped 0 0 0 0
17034 Error 0 0 0 0
17035 Last Flush: 5233 5535 5535
17036 Downstairs last five completed: 5233 5232 5231 5230 5229
17037 Upstairs last five completed: 5535 5534 5533 5532 5531
17038 CLIENT: Up:0 ds:302 act:2
17039 [8810] got start action
17040 [8810] I am going to restart
17041 Starting downstairs at port 8810
17042 Make output file at "/tmp/dsc/downstairs-8810.txt"
17043 [8810][0] reports Starting
17044 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1214)
17045 [8810][0] reports Running
17046 ----------------------------------------------------------------
17047 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17048 States: Offline Active Active
17049 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17050 5234 Acked Read 5 Sent Done Done false
17051 5235 Acked Write 8 Sent Done Done false
17052 5236 Acked Read 10 Sent Done Done false
17053 5237 Acked Read 1 Sent Done Done false
17054 5238 Acked Write 2 Sent Done Done false
17055 5239 Acked Flush 0 Sent Done Done false
17056 5240 Acked Write 8 Sent Done Done false
17057 5241 Acked Read 1 Sent Done Done false
17058 5242 Acked Write 8 Sent Done Done false
17059 5243 Acked Read 2 Sent Done Done false
17060 5244 Acked Read 6 Sent Done Done false
17061 5245 Acked Read 8 Sent Done Done false
17062 5246 Acked Read 1 Sent Done Done false
17063 5247 Acked Read 8 Sent Done Done false
17064 5248 Acked Read 8 Sent Done Done false
17065 5249 Acked Read 1 Sent Done Done false
17066 5250 Acked Flush 0 Sent Done Done false
17067 5251 Acked Read 7 Sent Done Done false
17068 5252 Acked Read 10 Sent Done Done false
17069 5253 Acked Read 4 Sent Done Done false
17070 5254 Acked Read 3 Sent Done Done false
17071 5255 Acked Flush 0 Sent Done Done false
17072 5256 Acked Write 10 Sent Done Done false
17073 5257 Acked Write 5 Sent Done Done false
17074 5258 Acked Read 9 Sent Done Done false
17075 5259 Acked Write 7 Sent Done Done false
17076 5260 Acked Flush 0 Sent Done Done false
17077 5261 Acked Read 6 Sent Done Done false
17078 5262 Acked Read 10 Sent Done Done false
17079 5263 Acked Write 4 Sent Done Done false
17080 5264 Acked Read 3 Sent Done Done false
17081 5265 Acked Read 9 Sent Done Done false
17082 5266 Acked Write 5 Sent Done Done false
17083 5267 Acked Read 7 Sent Done Done false
17084 5268 Acked Read 9 Sent Done Done false
17085 5269 Acked Flush 0 Sent Done Done false
17086 5270 Acked Write 9 Sent Done Done false
17087 5271 Acked Read 10 Sent Done Done false
17088 5272 Acked Read 2 Sent Done Done false
17089 5273 Acked Read 6 Sent Done Done false
17090 5274 Acked Read 7 Sent Done Done false
17091 5275 Acked Read 5 Sent Done Done false
17092 5276 Acked Read 2 Sent Done Done false
17093 5277 Acked Read 6 Sent Done Done false
17094 5278 Acked Read 2 Sent Done Done false
17095 5279 Acked Write 5 Sent Done Done false
17096 5280 Acked Write 9 Sent Done Done false
17097 5281 Acked Write 6 Sent Done Done false
17098 5282 Acked Read 5 Sent Done Done false
17099 5283 Acked Write 9 Sent Done Done false
17100 5284 Acked Write 7 Sent Done Done false
17101 5285 Acked Read 2 Sent Done Done false
17102 5286 Acked Write 2 Sent Done Done false
17103 5287 Acked Read 6 Sent Done Done false
17104 5288 Acked Read 3 Sent Done Done false
17105 5289 Acked Read 7 Sent Done Done false
17106 5290 Acked Write 8 Sent Done Done false
17107 5291 Acked Write 7 Sent Done Done false
17108 5292 Acked Write 8 Sent Done Done false
17109 5293 Acked Read 9 Sent Done Done false
17110 5294 Acked Write 5 Sent Done Done false
17111 5295 Acked Read 9 Sent Done Done false
17112 5296 Acked Read 5 Sent Done Done false
17113 5297 Acked Flush 0 Sent Done Done false
17114 5298 Acked Write 5 Sent Done Done false
17115 5299 Acked Read 4 Sent Done Done false
17116 5300 Acked Write 7 Sent Done Done false
17117 5301 Acked Write 6 Sent Done Done false
17118 5302 Acked Read 9 Sent Done Done false
17119 5303 Acked Read 8 Sent Done Done false
17120 5304 Acked Write 8 Sent Done Done false
17121 5305 Acked Write 3 Sent Done Done false
17122 5306 Acked Read 1 Sent Done Done false
17123 5307 Acked Read 8 Sent Done Done false
17124 5308 Acked Read 2 Sent Done Done false
17125 5309 Acked Write 6 Sent Done Done false
17126 5310 Acked Write 5 Sent Done Done false
17127 5311 Acked Read 6 Sent Done Done false
17128 5312 Acked Write 1 Sent Done Done false
17129 5313 Acked Read 2 Sent Done Done false
17130 5314 Acked Read 4 Sent Done Done false
17131 5315 Acked Write 8 Sent Done Done false
17132 5316 Acked Write 1 Sent Done Done false
17133 5317 Acked Write 3 Sent Done Done false
17134 5318 Acked Read 8 Sent Done Done false
17135 5319 Acked Write 3 Sent Done Done false
17136 5320 Acked Flush 0 Sent Done Done false
17137 5321 Acked Read 7 Sent Done Done false
17138 5322 Acked Write 2 Sent Done Done false
17139 5323 Acked Write 9 Sent Done Done false
17140 5324 Acked Write 2 Sent Done Done false
17141 5325 Acked Read 9 Sent Done Done false
17142 5326 Acked Write 10 Sent Done Done false
17143 5327 Acked Read 4 Sent Done Done false
17144 5328 Acked Read 1 Sent Done Done false
17145 5329 Acked Read 9 Sent Done Done false
17146 5330 Acked Read 1 Sent Done Done false
17147 5331 Acked Write 2 Sent Done Done false
17148 5332 Acked Read 4 Sent Done Done false
17149 5333 Acked Write 10 Sent Done Done false
17150 5334 Acked Read 6 Sent Done Done false
17151 5335 Acked Write 3 Sent Done Done false
17152 5336 Acked Read 7 Sent Done Done false
17153 5337 Acked Read 7 Sent Done Done false
17154 5338 Acked Flush 0 Sent Done Done false
17155 5339 Acked Read 9 Sent Done Done false
17156 5340 Acked Read 7 Sent Done Done false
17157 5341 Acked Read 10 Sent Done Done false
17158 5342 Acked Write 7 Sent Done Done false
17159 5343 Acked Flush 0 Sent Done Done false
17160 5344 Acked Flush 0 Sent Done Done false
17161 5345 Acked Write 8 Sent Done Done false
17162 5346 Acked Read 1 Sent Done Done false
17163 5347 Acked Read 6 Sent Done Done false
17164 5348 Acked Read 2 Sent Done Done false
17165 5349 Acked Write 1 Sent Done Done false
17166 5350 Acked Write 4 Sent Done Done false
17167 5351 Acked Write 5 Sent Done Done false
17168 5352 Acked Read 8 Sent Done Done false
17169 5353 Acked Read 1 Sent Done Done false
17170 5354 Acked Write 2 Sent Done Done false
17171 5355 Acked Read 10 Sent Done Done false
17172 5356 Acked Write 6 Sent Done Done false
17173 5357 Acked Read 4 Sent Done Done false
17174 5358 Acked Read 7 Sent Done Done false
17175 5359 Acked Write 9 Sent Done Done false
17176 5360 Acked Flush 0 Sent Done Done false
17177 5361 Acked Write 7 Sent Done Done false
17178 5362 Acked Write 9 Sent Done Done false
17179 5363 Acked Write 8 Sent Done Done false
17180 5364 Acked Read 3 Sent Done Done false
17181 5365 Acked Read 2 Sent Done Done false
17182 5366 Acked Write 9 Sent Done Done false
17183 5367 Acked Write 2 Sent Done Done false
17184 5368 Acked Read 1 Sent Done Done false
17185 5369 Acked Read 10 Sent Done Done false
17186 5370 Acked Read 7 Sent Done Done false
17187 5371 Acked Read 9 Sent Done Done false
17188 5372 Acked Read 7 Sent Done Done false
17189 5373 Acked Write 2 Sent Done Done false
17190 5374 Acked Read 3 Sent Done Done false
17191 5375 Acked Write 9 Sent Done Done false
17192 5376 Acked Write 8 Sent Done Done false
17193 5377 Acked Write 9 Sent Done Done false
17194 5378 Acked Write 9 Sent Done Done false
17195 5379 Acked Read 4 Sent Done Done false
17196 5380 Acked Read 5 Sent Done Done false
17197 5381 Acked Read 3 Sent Done Done false
17198 5382 Acked Write 10 Sent Done Done false
17199 5383 Acked Read 5 Sent Done Done false
17200 5384 Acked Flush 0 Sent Done Done false
17201 5385 Acked Write 7 Sent Done Done false
17202 5386 Acked Write 5 Sent Done Done false
17203 5387 Acked Read 2 Sent Done Done false
17204 5388 Acked Read 5 Sent Done Done false
17205 5389 Acked Write 9 Sent Done Done false
17206 5390 Acked Write 10 Sent Done Done false
17207 5391 Acked Read 10 Sent Done Done false
17208 5392 Acked Read 10 Sent Done Done false
17209 5393 Acked Write 4 Sent Done Done false
17210 5394 Acked Read 8 Sent Done Done false
17211 5395 Acked Write 5 Sent Done Done false
17212 5396 Acked Read 6 Sent Done Done false
17213 5397 Acked Read 8 Sent Done Done false
17214 5398 Acked Write 4 Sent Done Done false
17215 5399 Acked Read 2 Sent Done Done false
17216 5400 Acked Read 2 Sent Done Done false
17217 5401 Acked Write 1 Sent Done Done false
17218 5402 Acked Read 6 Sent Done Done false
17219 5403 Acked Read 5 Sent Done Done false
17220 5404 Acked Write 3 Sent Done Done false
17221 5405 Acked Read 8 Sent Done Done false
17222 5406 Acked Write 6 Sent Done Done false
17223 5407 Acked Read 1 Sent Done Done false
17224 5408 Acked Read 4 Sent Done Done false
17225 5409 Acked Flush 0 Sent Done Done false
17226 5410 Acked Write 1 Sent Done Done false
17227 5411 Acked Write 2 Sent Done Done false
17228 5412 Acked Write 3 Sent Done Done false
17229 5413 Acked Write 9 Sent Done Done false
17230 5414 Acked Write 8 Sent Done Done false
17231 5415 Acked Read 10 Sent Done Done false
17232 5416 Acked Write 1 Sent Done Done false
17233 5417 Acked Read 1 Sent Done Done false
17234 5418 Acked Read 4 Sent Done Done false
17235 5419 Acked Read 8 Sent Done Done false
17236 5420 Acked Read 3 Sent Done Done false
17237 5421 Acked Write 4 Sent Done Done false
17238 5422 Acked Read 5 Sent Done Done false
17239 5423 Acked Write 2 Sent Done Done false
17240 5424 Acked Write 5 Sent Done Done false
17241 5425 Acked Write 8 Sent Done Done false
17242 5426 Acked Read 2 Sent Done Done false
17243 5427 Acked Flush 0 Sent Done Done false
17244 5428 Acked Read 7 Sent Done Done false
17245 5429 Acked Write 9 Sent Done Done false
17246 5430 Acked Flush 0 Sent Done Done false
17247 5431 Acked Read 6 Sent Done Done false
17248 5432 Acked Write 6 Sent Done Done false
17249 5433 Acked Read 6 Sent Done Done false
17250 5434 Acked Write 10 Sent Done Done false
17251 5435 Acked Write 6 Sent Done Done false
17252 5436 Acked Read 3 Sent Done Done false
17253 5437 Acked Read 4 Sent Done Done false
17254 5438 Acked Read 2 Sent Done Done false
17255 5439 Acked Read 8 Sent Done Done false
17256 5440 Acked Write 7 Sent Done Done false
17257 5441 Acked Read 3 Sent Done Done false
17258 5442 Acked Write 6 Sent Done Done false
17259 5443 Acked Write 6 Sent Done Done false
17260 5444 Acked Read 7 Sent Done Done false
17261 5445 Acked Flush 0 Sent Done Done false
17262 5446 Acked Write 3 Sent Done Done false
17263 5447 Acked Read 6 Sent Done Done false
17264 5448 Acked Write 4 Sent Done Done false
17265 5449 Acked Read 6 Sent Done Done false
17266 5450 Acked Write 6 Sent Done Done false
17267 5451 Acked Write 1 Sent Done Done false
17268 5452 Acked Read 10 Sent Done Done false
17269 5453 Acked Flush 0 Sent Done Done false
17270 5454 Acked Write 8 Sent Done Done false
17271 5455 Acked Read 3 Sent Done Done false
17272 5456 Acked Write 3 Sent Done Done false
17273 5457 Acked Read 3 Sent Done Done false
17274 5458 Acked Read 9 Sent Done Done false
17275 5459 Acked Write 8 Sent Done Done false
17276 5460 Acked Write 5 Sent Done Done false
17277 5461 Acked Read 4 Sent Done Done false
17278 5462 Acked Read 9 Sent Done Done false
17279 5463 Acked Read 5 Sent Done Done false
17280 5464 Acked Read 4 Sent Done Done false
17281 5465 Acked Read 9 Sent Done Done false
17282 5466 Acked Read 4 Sent Done Done false
17283 5467 Acked Read 6 Sent Done Done false
17284 5468 Acked Read 6 Sent Done Done false
17285 5469 Acked Write 3 Sent Done Done false
17286 5470 Acked Write 5 Sent Done Done false
17287 5471 Acked Read 6 Sent Done Done false
17288 5472 Acked Write 1 Sent Done Done false
17289 5473 Acked Write 7 Sent Done Done false
17290 5474 Acked Write 9 Sent Done Done false
17291 5475 Acked Write 5 Sent Done Done false
17292 5476 Acked Read 6 Sent Done Done false
17293 5477 Acked Read 9 Sent Done Done false
17294 5478 Acked Write 10 Sent Done Done false
17295 5479 Acked Read 2 Sent Done Done false
17296 5480 Acked Flush 0 Sent Done Done false
17297 5481 Acked Write 3 Sent Done Done false
17298 5482 Acked Read 5 Sent Done Done false
17299 5483 Acked Write 3 Sent Done Done false
17300 5484 Acked Read 4 Sent Done Done false
17301 5485 Acked Write 5 Sent Done Done false
17302 5486 Acked Read 3 Sent Done Done false
17303 5487 Acked Write 1 Sent Done Done false
17304 5488 Acked Write 1 Sent Done Done false
17305 5489 Acked Write 4 Sent Done Done false
17306 5490 Acked Read 1 Sent Done Done false
17307 5491 Acked Read 8 Sent Done Done false
17308 5492 Acked Write 5 Sent Done Done false
17309 5493 Acked Write 10 Sent Done Done false
17310 5494 Acked Read 10 Sent Done Done false
17311 5495 Acked Read 6 Sent Done Done false
17312 5496 Acked Write 9 Sent Done Done false
17313 5497 Acked Write 8 Sent Done Done false
17314 5498 Acked Write 5 Sent Done Done false
17315 5499 Acked Flush 0 Sent Done Done false
17316 5500 Acked Read 7 Sent Done Done false
17317 5501 Acked Read 3 Sent Done Done false
17318 5502 Acked Read 10 Sent Done Done false
17319 5503 Acked Flush 0 Sent Done Done false
17320 5504 Acked Flush 0 Sent Done Done false
17321 5505 Acked Read 1 Sent Done Done false
17322 5506 Acked Read 2 Sent Done Done false
17323 5507 Acked Write 5 Sent Done Done false
17324 5508 Acked Read 5 Sent Done Done false
17325 5509 Acked Read 8 Sent Done Done false
17326 5510 Acked Write 5 Sent Done Done false
17327 5511 Acked Flush 0 Sent Done Done false
17328 5512 Acked Flush 0 Sent Done Done false
17329 5513 Acked Read 2 Sent Done Done false
17330 5514 Acked Flush 0 Sent Done Done false
17331 5515 Acked Write 4 Sent Done Done false
17332 5516 Acked Read 7 Sent Done Done false
17333 5517 Acked Flush 0 Sent Done Done false
17334 5518 Acked Write 7 Sent Done Done false
17335 5519 Acked Write 8 Sent Done Done false
17336 5520 Acked Read 7 Sent Done Done false
17337 5521 Acked Read 9 Sent Done Done false
17338 5522 Acked Read 10 Sent Done Done false
17339 5523 Acked Write 2 Sent Done Done false
17340 5524 Acked Read 3 Sent Done Done false
17341 5525 Acked Flush 0 Sent Done Done false
17342 5526 Acked Write 9 Sent Done Done false
17343 5527 Acked Write 4 Sent Done Done false
17344 5528 Acked Read 8 Sent Done Done false
17345 5529 Acked Write 4 Sent Done Done false
17346 5530 Acked Write 5 Sent Done Done false
17347 5531 Acked Read 9 Sent Done Done false
17348 5532 Acked Write 5 Sent Done Done false
17349 5533 Acked Write 1 Sent Done Done false
17350 5534 Acked Flush 0 Sent Done Done false
17351 5535 Acked Flush 0 Sent Done Done false
17352 STATES DS:0 DS:1 DS:2 TOTAL
17353 Sent 302 0 0 302
17354 Done 0 302 302 604
17355 Skipped 0 0 0 0
17356 Error 0 0 0 0
17357 Last Flush: 5233 5535 5535
17358 Downstairs last five completed: 5233 5232 5231 5230 5229
17359 Upstairs last five completed: 5535 5534 5533 5532 5531
17360 CLIENT: Up:0 ds:302 act:2
173612024-11-07T14:06:55.962ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173622024-11-07T14:06:55.962ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43162) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173632024-11-07T14:06:55.963ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173642024-11-07T14:06:55.963ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173652024-11-07T14:06:55.963ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173662024-11-07T14:06:55.963ZINFOcrucible: send last flush ID to this DS: 5233 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173672024-11-07T14:06:55.964ZINFOcrucible: Replied this last flush ID: 5233 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173682024-11-07T14:06:55.964ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173692024-11-07T14:06:55.964ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173702024-11-07T14:06:55.964ZINFOcrucible: [0] client re-new 302 jobs since flush 5233 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173712024-11-07T14:06:55.964ZINFOcrucible: [0] Replayed 302 jobs since flush: 5233 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
17372 ----------------------------------------------------------------
17373 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17374 Downstairs last five completed: 5535 5534 5533 5532 5531
17375 Upstairs last five completed: 5535 5534 5533 5532 5531
17376 CLIENT: Up:0 ds:0 act:3
17377 Replay: All jobs finished
17378 Main task has work to do, go find it
17379 got dsc Stop(2)
17380 stop 2
17381 [8830] Got stop action so:false kr:false
17382 Nov 07 14:06:58.257 INFO request completed, latency_us: 341, response_code: 204, uri: /stop/cid/2, method: POST, req_id: e07e0821-6e5c-432d-97f2-a5b25783f4eb, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
173832024-11-07T14:06:58.258ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173842024-11-07T14:06:58.258ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173852024-11-07T14:06:58.258ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
17386 Nov 07 14:06:58.258 INFO request completed, latency_us: 119, response_code: 200, uri: {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:06:58.258392856Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"client":"2","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
17387 /state/cid/2, method: GET, req_id: ddd8e691-c873-43b8-a3bf-be48bf90237d, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
173882024-11-07T14:06:58.258ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
173892024-11-07T14:06:58.258ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
17390 [8830] Exited with: ExitStatus(unix_wait_status(9))
17391 [8830][2] reports Exit
17392 Nov 07 14:07:02.260 INFO request completed, latency_us: 200, response_code: 200, uri: /state/cid/2, method: GET, req_id: a3ef94a1-e7a6-461a-ae87-a487d722604e, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
17393 001/300 Read block 917 len 20480
17394 002/300 Read block 651 len 40960
17395 003/300 Read block 3863 len 8192
17396 004/300 Read block 3205 len 40960
17397 005/300 Read block 1772 len 20480
17398 006/300 Read block 4643 len 36864
17399 007/300 Read block 2585 len 8192
17400 008/300 Write block 885 len 16384 data: 4 3 4 4
17401 009/300 Read block 4294 len 16384
17402 010/300 Read block 4851 len 36864
17403 011/300 Write block 4924 len 32768 data: 4 4 5 4 4 4 4 4
17404 012/300 Read block 1396 len 4096
17405 013/300 Read block 1304 len 32768
17406 014/300 Write block 4416 len 36864 data: 3 3 2 2 2 2 2 2 2
17407 015/300 Read block 332 len 36864
17408 016/300 Write block 4482 len 40960 data: 7 6 5 6 5 5 4 2 4 4
17409 017/300 Write block 1857 len 16384 data: 5 5 4 4
17410 018/300 Read block 1904 len 40960
17411 019/300 Write block 662 len 20480 data: 3 3 2 2 2
17412 020/300 Flush
17413 021/300 Write block 3375 len 12288 data: 5 7 8
17414 022/300 Write block 3318 len 12288 data: 3 2 2
17415 023/300 Read block 1986 len 4096
17416 024/300 Write block 1223 len 8192 data: 4 4
17417 025/300 Read block 2303 len 8192
17418 026/300 Read block 749 len 32768
17419 027/300 Flush
17420 028/300 Write block 4253 len 40960 data: 4 4 4 4 3 4 3 4 5 4
17421 029/300 Read block 2967 len 4096
17422 030/300 Write block 708 len 36864 data: 6 5 4 3 3 3 4 3 2
17423 031/300 Flush
17424 032/300 Read block 855 len 20480
17425 033/300 Read block 876 len 8192
17426 034/300 Read block 2330 len 24576
17427 035/300 Read block 4271 len 20480
17428 036/300 Read block 2146 len 12288
17429 037/300 Read block 4719 len 28672
17430 038/300 Read block 4327 len 16384
17431 039/300 Read block 1961 len 24576
17432 040/300 Read block 2115 len 20480
17433 041/300 Read block 3158 len 8192
17434 042/300 Flush
17435 043/300 Write block 1440 len 20480 data: 4 4 7 7 6
17436 044/300 Read block 1721 len 40960
17437 045/300 Write block 142 len 40960 data: 4 4 4 4 5 6 6 6 4 5
17438 046/300 Write block 3267 len 36864 data: 6 3 5 5 5 6 7 6 5
17439 047/300 Write block 4391 len 36864 data: 6 4 4 3 3 3 3 3 3
17440 048/300 Flush
17441 049/300 Write block 2803 len 28672 data: 4 3 4 4 4 4 5
17442 050/300 Read block 303 len 12288
17443 051/300 Write block 209 len 24576 data: 3 3 3 3 3 3
17444 052/300 Write block 26 len 16384 data: 7 7 7 5
17445 053/300 Write block 631 len 8192 data: 3 4
17446 054/300 Read block 512 len 20480
17447 055/300 Read block 3696 len 40960
17448 056/300 Write block 799 len 12288 data: 3 2 2
17449 057/300 Write block 4397 len 8192 data: 4 4
17450 058/300 Write block 2347 len 12288 data: 2 2 2
17451 059/300 Read block 2703 len 32768
17452 060/300 Write block 4556 len 40960 data: 6 6 7 8 5 3 3 2 2 2
17453 061/300 Read block 1813 len 40960
17454 062/300 Write block 254 len 32768 data: 3 3 5 3 3 4 4 4
17455 063/300 Read block 752 len 32768
17456 064/300 Read block 1133 len 36864
17457 065/300 Write block 2619 len 12288 data: 5 6 4
17458 066/300 Read block 4860 len 20480
17459 067/300 Write block 3106 len 24576 data: 4 4 4 3 2 2
17460 068/300 Read block 862 len 28672
17461 069/300 Read block 4762 len 8192
17462 070/300 Write block 2277 len 40960 data: 8 6 6 5 3 3 3 2 2 2
17463 071/300 Write block 2887 len 8192 data: 5 5
17464 072/300 Read block 3885 len 16384
17465 073/300 Write block 3883 len 36864 data: 7 6 4 4 4 4 5 5 4
17466 074/300 Flush
17467 075/300 Write block 1257 len 24576 data: 3 3 3 3 2 2
17468 076/300 Read block 2741 len 32768
17469 077/300 Read block 3342 len 8192
17470 078/300 Read block 1882 len 36864
17471 079/300 Read block 4641 len 12288
17472 080/300 Read block 2745 len 28672
17473 081/300 Write block 1050 len 12288 data: 5 5 5
17474 082/300 Read block 91 len 12288
17475 083/300 Read block 4885 len 24576
17476 084/300 Write block 4763 len 40960 data: 4 4 5 4 3 2 2 3 4 4
17477 085/300 Read block 1224 len 20480
17478 086/300 Read block 2869 len 40960
17479 087/300 Write block 4421 len 36864 data: 3 3 3 3 2 2 2 2 2
17480 088/300 Read block 3426 len 8192
17481 089/300 Read block 3521 len 28672
17482 090/300 Write block 1580 len 32768 data: 5 4 3 3 3 3 4 5
17483 091/300 Write block 1950 len 28672 data: 6 4 3 2 3 3 3
17484 092/300 Write block 403 len 36864 data: 6 7 7 6 5 5 5 6 6
17485 093/300 Read block 4147 len 36864
17486 094/300 Write block 1492 len 20480 data: 3 4 4 3 3
17487 095/300 Read block 3376 len 40960
17488 096/300 Write block 2185 len 36864 data: 6 5 4 3 3 3 3 3 4
17489 097/300 Write block 179 len 8192 data: 4 4
17490 098/300 Flush
17491 099/300 Read block 429 len 20480
17492 100/300 Read block 2596 len 8192
17493 101/300 Write block 3223 len 16384 data: 3 3 2 2
17494 102/300 Read block 2402 len 24576
17495 103/300 Write block 4141 len 8192 data: 5 3
17496 104/300 Read block 4064 len 36864
17497 105/300 Flush
17498 106/300 Write block 3962 len 28672 data: 3 4 4 6 5 5 5
17499 107/300 Read block 948 len 12288
17500 108/300 Read block 2542 len 36864
17501 109/300 Read block 3334 len 24576
17502 110/300 Write block 1116 len 32768 data: 3 3 3 3 3 3 3 3
17503 111/300 Write block 2164 len 16384 data: 6 7 7 7
17504 112/300 Read block 4870 len 16384
17505 113/300 Flush
17506 114/300 Read block 3356 len 12288
17507 115/300 Write block 24 len 40960 data: 5 6 8 8 8 6 4 4 4 4
17508 116/300 Read block 3007 len 32768
17509 117/300 Write block 1480 len 28672 data: 4 4 2 2 2 2 2
17510 118/300 Write block 4720 len 24576 data: 3 3 3 4 5 5
17511 119/300 Read block 1020 len 28672
17512 120/300 Flush
17513 121/300 Read block 4801 len 28672
17514 122/300 Write block 4450 len 8192 data: 5 3
17515 123/300 Write block 3860 len 40960 data: 5 5 7 7 7 6 5 5 5 6
17516 124/300 Write block 3330 len 24576 data: 4 6 6 5 5 5
17517 125/300 Write block 3486 len 40960 data: 5 5 6 4 4 4 5 4 4 4
17518 126/300 Flush
17519 127/300 Flush
17520 128/300 Write block 2312 len 4096 data: 4
17521 129/300 Read block 1620 len 16384
17522 130/300 Write block 1601 len 28672 data: 7 7 7 7 6 5 4
17523 131/300 Write block 568 len 8192 data: 5 5
17524 132/300 Write block 33 len 20480 data: 5 3 2 2 2
17525 133/300 Write block 2447 len 28672 data: 3 3 3 4 4 4 4
17526 134/300 Write block 1183 len 8192 data: 3 3
17527 135/300 Read block 414 len 8192
17528 136/300 Write block 4234 len 12288 data: 4 4 3
17529 137/300 Write block 751 len 4096 data: 5
17530 138/300 Write block 205 len 28672 data: 2 3 3 3 4 4 4
17531 139/300 Write block 3288 len 40960 data: 3 4 4 3 6 5 5 6 6 6
17532 140/300 Flush
17533 141/300 Read block 3608 len 4096
17534 142/300 Read block 2479 len 24576
17535 143/300 Write block 1185 len 20480 data: 3 4 5 6 6
17536 144/300 Write block 1482 len 32768 data: 3 3 3 3 3 2 2 2
17537 145/300 Write block 4166 len 36864 data: 2 3 3 3 3 5 5 6 7
17538 146/300 Read block 4047 len 20480
17539 147/300 Flush
17540 148/300 Write block 4893 len 40960 data: 4 5 6 7 5 4 4 4 2 2
17541 149/300 Read block 4357 len 12288
17542 150/300 Read block 320 len 40960
17543 151/300 Write block 3285 len 16384 data: 5 4 3 4
17544 152/300 Read block 1709 len 20480
17545 153/300 Flush
17546 154/300 Write block 2104 len 36864 data: 4 3 4 4 3 2 2 2 3
17547 155/300 Write block 3341 len 28672 data: 4 5 4 5 6 5 4
17548 156/300 Read block 701 len 40960
17549 157/300 Read block 3218 len 16384
17550 158/300 Read block 3848 len 32768
17551 159/300 Read block 3945 len 36864
17552 160/300 Write block 2549 len 16384 data: 5 4 5 4
17553 161/300 Read block 4764 len 40960
17554 162/300 Write block 2474 len 20480 data: 5 6 6 5 5
17555 163/300 Read block 1513 len 16384
17556 164/300 Read block 1406 len 40960
17557 165/300 Read block 123 len 8192
17558 166/300 Write block 3646 len 12288 data: 3 3 3
17559 167/300 Write block 619 len 24576 data: 2 2 2 3 4 3
17560 168/300 Read block 3338 len 24576
17561 169/300 Read block 4901 len 4096
17562 170/300 Read block 3607 len 4096
17563 171/300 Read block 1069 len 32768
17564 172/300 Read block 1579 len 16384
17565 173/300 Write block 1343 len 32768 data: 4 3 3 3 4 4 5 4
17566 174/300 Read block 54 len 16384
17567 175/300 Write block 1289 len 8192 data: 2 2
17568 176/300 Read block 4846 len 24576
17569 177/300 Read block 3029 len 40960
17570 178/300 Flush
17571 179/300 Read block 797 len 24576
17572 180/300 Read block 1372 len 40960
17573 181/300 Flush
17574 182/300 Read block 1474 len 40960
17575 183/300 Write block 4392 len 16384 data: 5 5 4 4
17576 184/300 Flush
17577 185/300 Read block 1895 len 36864
17578 186/300 Read block 1667 len 20480
17579 187/300 Write block 3985 len 20480 data: 4 4 4 4 4
17580 188/300 Flush
17581 189/300 Read block 191 len 24576
17582 190/300 Read block 2140 len 36864
17583 191/300 Flush
17584 192/300 Write block 3305 len 40960 data: 4 4 4 5 5 5 5 5 4 3
17585 193/300 Write block 114 len 32768 data: 2 2 2 2 2 2 2 2
17586 194/300 Write block 2455 len 16384 data: 4 4 4 4
17587 195/300 Read block 4492 len 36864
17588 196/300 Read block 1825 len 24576
17589 197/300 Read block 688 len 12288
17590 198/300 Read block 3372 len 4096
17591 199/300 Read block 3854 len 32768
17592 200/300 Write block 2293 len 8192 data: 2 2
17593 201/300 Flush
17594 202/300 Read block 1075 len 12288
17595 203/300 Read block 4807 len 32768
17596 204/300 Read block 2830 len 40960
17597 205/300 Write block 3373 len 20480 data: 3 3 6 8 9
17598 206/300 Write block 3739 len 28672 data: 5 3 3 3 4 4 3
17599 207/300 Write block 4178 len 36864 data: 5 4 3 3 3 3 3 3 2
17600 208/300 Write block 4015 len 16384 data: 6 5 5 5
17601 209/300 Flush
17602 210/300 Read block 3584 len 36864
17603 211/300 Write block 276 len 8192 data: 3 3
17604 212/300 Read block 2464 len 40960
17605 213/300 Read block 1328 len 28672
17606 214/300 Write block 4551 len 12288 data: 2 2 3
17607 215/300 Write block 1590 len 20480 data: 4 4 4 4 3
17608 216/300 Write block 1822 len 36864 data: 3 3 4 4 3 4 5 4 5
17609 217/300 Read block 2990 len 12288
17610 218/300 Flush
17611 219/300 Write block 2866 len 8192 data: 3 3
17612 220/300 Read block 1017 len 12288
17613 221/300 Write block 3186 len 16384 data: 2 2 2 2
17614 222/300 Read block 855 len 32768
17615 223/300 Read block 1544 len 8192
17616 224/300 Write block 1498 len 8192 data: 4 4
17617 225/300 Read block 3403 len 4096
17618 226/300 Read block 4147 len 12288
17619 227/300 Write block 3373 len 8192 data: 4 4
17620 228/300 Write block 4536 len 12288 data: 5 5 6
17621 229/300 Read block 898 len 32768
17622 230/300 Read block 2418 len 4096
17623 231/300 Read block 1695 len 4096
17624 232/300 Read block 445 len 40960
17625 233/300 Flush
17626 234/300 Write block 3996 len 40960 data: 3 3 4 5 5 5 5 5 5 4
17627 235/300 Read block 4914 len 12288
17628 236/300 Read block 905 len 40960
17629 237/300 Read block 1350 len 40960
17630 238/300 Write block 4218 len 4096 data: 2
17631 239/300 Write block 3338 len 8192 data: 5 5
17632 240/300 Read block 4066 len 8192
17633 241/300 Flush
17634 242/300 Flush
17635 243/300 Write block 2970 len 20480 data: 4 4 4 4 4
17636 244/300 Write block 544 len 36864 data: 4 4 4 5 4 3 3 3 3
17637 245/300 Flush
17638 246/300 Read block 3486 len 20480
17639 247/300 Flush
17640 248/300 Write block 1647 len 12288 data: 6 7 6
17641 249/300 Write block 2777 len 36864 data: 2 2 2 3 3 4 4 4 4
17642 250/300 Flush
17643 251/300 Read block 2710 len 28672
17644 252/300 Flush
17645 253/300 Read block 4958 len 8192
17646 254/300 Write block 4076 len 12288 data: 4 4 4
17647 255/300 Write block 1892 len 4096 data: 3
17648 256/300 Write block 635 len 8192 data: 4 4
17649 257/300 Flush
17650 258/300 Write block 1282 len 40960 data: 4 4 4 4 4 2 2 3 3 2
17651 259/300 Flush
17652 260/300 Read block 770 len 24576
17653 261/300 Read block 1373 len 28672
17654 262/300 Read block 748 len 32768
17655 263/300 Read block 2260 len 12288
17656 264/300 Write block 385 len 32768 data: 3 3 3 2 2 2 2 2
17657 265/300 Read block 1244 len 16384
17658 266/300 Read block 3335 len 8192
17659 267/300 Write block 1507 len 28672 data: 5 4 5 3 3 3 2
17660 268/300 Write block 1216 len 40960 data: 4 4 4 4 3 3 5 5 5 4
17661 269/300 Read block 4378 len 40960
17662 270/300 Write block 4371 len 36864 data: 6 4 3 3 4 3 3 3 3
17663 271/300 Write block 1998 len 24576 data: 4 4 4 4 4 2
17664 272/300 Read block 2789 len 16384
17665 273/300 Read block 2510 len 16384
17666 274/300 Write block 854 len 36864 data: 4 4 4 4 3 3 3 3 3
17667 275/300 Write block 3283 len 4096 data: 5
17668 276/300 Read block 4114 len 12288
17669 277/300 Write block 3950 len 40960 data: 2 3 3 3 3 3 4 3 3 3
17670 278/300 Write block 2448 len 28672 data: 4 4 5 5 5 5 4
17671 279/300 Read block 2440 len 36864
17672 280/300 Write block 2324 len 40960 data: 4 3 3 5 5 5 4 4 4 4
17673 281/300 Flush
17674 282/300 Read block 3877 len 8192
17675 283/300 Read block 1182 len 12288
17676 284/300 Write block 1805 len 16384 data: 3 4 4 4
17677 285/300 Read block 2403 len 8192
17678 286/300 Read block 465 len 4096
17679 287/300 Read block 4086 len 24576
17680 288/300 Write block 3054 len 4096 data: 5
17681 289/300 Write block 2774 len 16384 data: 2 2 2 3
17682 290/300 Read block 61 len 12288
17683 291/300 Read block 3700 len 20480
17684 292/300 Read block 3201 len 32768
17685 293/300 Write block 3292 len 40960 data: 7 6 6 7 7 7 6 3 3 3
17686 294/300 Write block 4643 len 24576 data: 5 4 4 4 5 5
17687 295/300 Write block 1373 len 20480 data: 4 6 5 5 4
17688 296/300 Read block 4297 len 32768
17689 297/300 Flush
17690 298/300 Flush
17691 299/300 Read block 2895 len 4096
17692 300/300 Write block 2275 len 20480 data: 5 5 9 7 7
17693 301/300 Read block 1149 len 8192
17694 Main task has work to do, go find it
17695 got dsc Start(2)
17696 start 2
17697 Nov 07 14:07:02.550 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/2, method: POST, req_id: eaf41f47-cd0b-4c01-b634-a9b737f9863c, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
17698 Replay: started 2, returned:Ok(())
17699 ----------------------------------------------------------------
17700 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17701 States: Active Active Offline
17702 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17703 5536 Acked Read 5 Done Done Sent false
17704 5537 Acked Read 10 Done Done Sent false
17705 5538 Acked Read 2 Done Done Sent false
17706 5539 Acked Read 10 Done Done Sent false
17707 5540 Acked Read 5 Done Done Sent false
17708 5541 Acked Read 9 Done Done Sent false
17709 5542 Acked Read 2 Done Done Sent false
17710 5543 Acked Write 4 Done Done Sent false
17711 5544 Acked Read 4 Done Done Sent false
17712 5545 Acked Read 9 Done Done Sent false
17713 5546 Acked Write 8 Done Done Sent false
17714 5547 Acked Read 1 Done Done Sent false
17715 5548 Acked Read 8 Done Done Sent false
17716 5549 Acked Write 9 Done Done Sent false
17717 5550 Acked Read 9 Done Done Sent false
17718 5551 Acked Write 10 Done Done Sent false
17719 5552 Acked Write 4 Done Done Sent false
17720 5553 Acked Read 10 Done Done Sent false
17721 5554 Acked Write 5 Done Done Sent false
17722 5555 Acked Flush 0 Done Done Sent false
17723 5556 Acked Write 3 Done Done Sent false
17724 5557 Acked Write 3 Done Done Sent false
17725 5558 Acked Read 1 Done Done Sent false
17726 5559 Acked Write 2 Done Done Sent false
17727 5560 Acked Read 2 Done Done Sent false
17728 5561 Acked Read 8 Done Done Sent false
17729 5562 Acked Flush 0 Done Done Sent false
17730 5563 Acked Write 10 Done Done Sent false
17731 5564 Acked Read 1 Done Done Sent false
17732 5565 Acked Write 9 Done Done Sent false
17733 5566 Acked Flush 0 Done Done Sent false
17734 5567 Acked Read 5 Done Done Sent false
17735 5568 Acked Read 2 Done Done Sent false
17736 5569 Acked Read 6 Done Done Sent false
17737 5570 Acked Read 5 Done Done Sent false
17738 5571 Acked Read 3 Done Done Sent false
17739 5572 Acked Read 7 Done Done Sent false
17740 5573 Acked Read 4 Done Done Sent false
17741 5574 Acked Read 6 Done Done Sent false
17742 5575 Acked Read 5 Done Done Sent false
17743 5576 Acked Read 2 Done Done Sent false
17744 5577 Acked Flush 0 Done Done Sent false
17745 5578 Acked Write 5 Done Done Sent false
17746 5579 Acked Read 10 Done Done Sent false
17747 5580 Acked Write 10 Done Done Sent false
17748 5581 Acked Write 9 Done Done Sent false
17749 5582 Acked Write 9 Done Done Sent false
17750 5583 Acked Flush 0 Done Done Sent false
17751 5584 Acked Write 7 Done Done Sent false
17752 5585 Acked Read 3 Done Done Sent false
17753 5586 Acked Write 6 Done Done Sent false
17754 5587 Acked Write 4 Done Done Sent false
17755 5588 Acked Write 2 Done Done Sent false
17756 5589 Acked Read 5 Done Done Sent false
17757 5590 Acked Read 10 Done Done Sent false
17758 5591 Acked Write 3 Done Done Sent false
17759 5592 Acked Write 2 Done Done Sent false
17760 5593 Acked Write 3 Done Done Sent false
17761 5594 Acked Read 8 Done Done Sent false
17762 5595 Acked Write 10 Done Done Sent false
17763 5596 Acked Read 10 Done Done Sent false
17764 5597 Acked Write 8 Done Done Sent false
17765 5598 Acked Read 8 Done Done Sent false
17766 5599 Acked Read 9 Done Done Sent false
17767 5600 Acked Write 3 Done Done Sent false
17768 5601 Acked Read 5 Done Done Sent false
17769 5602 Acked Write 6 Done Done Sent false
17770 5603 Acked Read 7 Done Done Sent false
17771 5604 Acked Read 2 Done Done Sent false
17772 5605 Acked Write 10 Done Done Sent false
17773 5606 Acked Write 2 Done Done Sent false
17774 5607 Acked Read 4 Done Done Sent false
17775 5608 Acked Write 9 Done Done Sent false
17776 5609 Acked Flush 0 Done Done Sent false
17777 5610 Acked Write 6 Done Done Sent false
17778 5611 Acked Read 8 Done Done Sent false
17779 5612 Acked Read 2 Done Done Sent false
17780 5613 Acked Read 9 Done Done Sent false
17781 5614 Acked Read 3 Done Done Sent false
17782 5615 Acked Read 7 Done Done Sent false
17783 5616 Acked Write 3 Done Done Sent false
17784 5617 Acked Read 3 Done Done Sent false
17785 5618 Acked Read 6 Done Done Sent false
17786 5619 Acked Write 10 Done Done Sent false
17787 5620 Acked Read 5 Done Done Sent false
17788 5621 Acked Read 10 Done Done Sent false
17789 5622 Acked Write 9 Done Done Sent false
17790 5623 Acked Read 2 Done Done Sent false
17791 5624 Acked Read 7 Done Done Sent false
17792 5625 Acked Write 8 Done Done Sent false
17793 5626 Acked Write 7 Done Done Sent false
17794 5627 Acked Write 9 Done Done Sent false
17795 5628 Acked Read 9 Done Done Sent false
17796 5629 Acked Write 5 Done Done Sent false
17797 5630 Acked Read 10 Done Done Sent false
17798 5631 Acked Write 9 Done Done Sent false
17799 5632 Acked Write 2 Done Done Sent false
17800 5633 Acked Flush 0 Done Done Sent false
17801 5634 Acked Read 5 Done Done Sent false
17802 5635 Acked Read 2 Done Done Sent false
17803 5636 Acked Write 4 Done Done Sent false
17804 5637 Acked Read 6 Done Done Sent false
17805 5638 Acked Write 2 Done Done Sent false
17806 5639 Acked Read 9 Done Done Sent false
17807 5640 Acked Flush 0 Done Done Sent false
17808 5641 Acked Write 7 Done Done Sent false
17809 5642 Acked Read 3 Done Done Sent false
17810 5643 Acked Read 9 Done Done Sent false
17811 5644 Acked Read 6 Done Done Sent false
17812 5645 Acked Write 8 Done Done Sent false
17813 5646 Acked Write 4 Done Done Sent false
17814 5647 Acked Read 4 Done Done Sent false
17815 5648 Acked Flush 0 Done Done Sent false
17816 5649 Acked Read 3 Done Done Sent false
17817 5650 Acked Write 10 Done Done Sent false
17818 5651 Acked Read 8 Done Done Sent false
17819 5652 Acked Write 7 Done Done Sent false
17820 5653 Acked Write 6 Done Done Sent false
17821 5654 Acked Read 7 Done Done Sent false
17822 5655 Acked Flush 0 Done Done Sent false
17823 5656 Acked Read 7 Done Done Sent false
17824 5657 Acked Write 2 Done Done Sent false
17825 5658 Acked Write 10 Done Done Sent false
17826 5659 Acked Write 6 Done Done Sent false
17827 5660 Acked Write 10 Done Done Sent false
17828 5661 Acked Flush 0 Done Done Sent false
17829 5662 Acked Flush 0 Done Done Sent false
17830 5663 Acked Write 1 Done Done Sent false
17831 5664 Acked Read 4 Done Done Sent false
17832 5665 Acked Write 7 Done Done Sent false
17833 5666 Acked Write 2 Done Done Sent false
17834 5667 Acked Write 5 Done Done Sent false
17835 5668 Acked Write 7 Done Done Sent false
17836 5669 Acked Write 2 Done Done Sent false
17837 5670 Acked Read 2 Done Done Sent false
17838 5671 Acked Write 3 Done Done Sent false
17839 5672 Acked Write 1 Done Done Sent false
17840 5673 Acked Write 7 Done Done Sent false
17841 5674 Acked Write 10 Done Done Sent false
17842 5675 Acked Flush 0 Done Done Sent false
17843 5676 Acked Read 1 Done Done Sent false
17844 5677 Acked Read 6 Done Done Sent false
17845 5678 Acked Write 5 Done Done Sent false
17846 5679 Acked Write 8 Done Done Sent false
17847 5680 Acked Write 9 Done Done Sent false
17848 5681 Acked Read 5 Done Done Sent false
17849 5682 Acked Flush 0 Done Done Sent false
17850 5683 Acked Write 10 Done Done Sent false
17851 5684 Acked Read 3 Done Done Sent false
17852 5685 Acked Read 10 Done Done Sent false
17853 5686 Acked Write 4 Done Done Sent false
17854 5687 Acked Read 5 Done Done Sent false
17855 5688 Acked Flush 0 Done Done Sent false
17856 5689 Acked Write 9 Done Done Sent false
17857 5690 Acked Write 7 Done Done Sent false
17858 5691 Acked Read 10 Done Done Sent false
17859 5692 Acked Read 4 Done Done Sent false
17860 5693 Acked Read 8 Done Done Sent false
17861 5694 Acked Read 9 Done Done Sent false
17862 5695 Acked Write 4 Done Done Sent false
17863 5696 Acked Read 10 Done Done Sent false
17864 5697 Acked Write 5 Done Done Sent false
17865 5698 Acked Read 4 Done Done Sent false
17866 5699 Acked Read 10 Done Done Sent false
17867 5700 Acked Read 2 Done Done Sent false
17868 5701 Acked Write 3 Done Done Sent false
17869 5702 Acked Write 6 Done Done Sent false
17870 5703 Acked Read 6 Done Done Sent false
17871 5704 Acked Read 1 Done Done Sent false
17872 5705 Acked Read 1 Done Done Sent false
17873 5706 Acked Read 8 Done Done Sent false
17874 5707 Acked Read 4 Done Done Sent false
17875 5708 Acked Write 8 Done Done Sent false
17876 5709 Acked Read 4 Done Done Sent false
17877 5710 Acked Write 2 Done Done Sent false
17878 5711 Acked Read 6 Done Done Sent false
17879 5712 Acked Read 10 Done Done Sent false
17880 5713 Acked Flush 0 Done Done Sent false
17881 5714 Acked Read 6 Done Done Sent false
17882 5715 Acked Read 10 Done Done Sent false
17883 5716 Acked Flush 0 Done Done Sent false
17884 5717 Acked Read 10 Done Done Sent false
17885 5718 Acked Write 4 Done Done Sent false
17886 5719 Acked Flush 0 Done Done Sent false
17887 5720 Acked Read 9 Done Done Sent false
17888 5721 Acked Read 5 Done Done Sent false
17889 5722 Acked Write 5 Done Done Sent false
17890 5723 Acked Flush 0 Done Done Sent false
17891 5724 Acked Read 6 Done Done Sent false
17892 5725 Acked Read 9 Done Done Sent false
17893 5726 Acked Flush 0 Done Done Sent false
17894 5727 Acked Write 10 Done Done Sent false
17895 5728 Acked Write 8 Done Done Sent false
17896 5729 Acked Write 4 Done Done Sent false
17897 5730 Acked Read 9 Done Done Sent false
17898 5731 Acked Read 6 Done Done Sent false
17899 5732 Acked Read 3 Done Done Sent false
17900 5733 Acked Read 1 Done Done Sent false
17901 5734 Acked Read 8 Done Done Sent false
17902 5735 Acked Write 2 Done Done Sent false
17903 5736 Acked Flush 0 Done Done Sent false
17904 5737 Acked Read 3 Done Done Sent false
17905 5738 Acked Read 8 Done Done Sent false
17906 5739 Acked Read 10 Done Done Sent false
17907 5740 Acked Write 5 Done Done Sent false
17908 5741 Acked Write 7 Done Done Sent false
17909 5742 Acked Write 9 Done Done Sent false
17910 5743 Acked Write 4 Done Done Sent false
17911 5744 Acked Flush 0 Done Done Sent false
17912 5745 Acked Read 9 Done Done Sent false
17913 5746 Acked Write 2 Done Done Sent false
17914 5747 Acked Read 10 Done Done Sent false
17915 5748 Acked Read 7 Done Done Sent false
17916 5749 Acked Write 3 Done Done Sent false
17917 5750 Acked Write 5 Done Done Sent false
17918 5751 Acked Write 9 Done Done Sent false
17919 5752 Acked Read 3 Done Done Sent false
17920 5753 Acked Flush 0 Done Done Sent false
17921 5754 Acked Write 2 Done Done Sent false
17922 5755 Acked Read 3 Done Done Sent false
17923 5756 Acked Write 4 Done Done Sent false
17924 5757 Acked Read 8 Done Done Sent false
17925 5758 Acked Read 2 Done Done Sent false
17926 5759 Acked Write 2 Done Done Sent false
17927 5760 Acked Read 1 Done Done Sent false
17928 5761 Acked Read 3 Done Done Sent false
17929 5762 Acked Write 2 Done Done Sent false
17930 5763 Acked Write 3 Done Done Sent false
17931 5764 Acked Read 8 Done Done Sent false
17932 5765 Acked Read 1 Done Done Sent false
17933 5766 Acked Read 1 Done Done Sent false
17934 5767 Acked Read 10 Done Done Sent false
17935 5768 Acked Flush 0 Done Done Sent false
17936 5769 Acked Write 10 Done Done Sent false
17937 5770 Acked Read 3 Done Done Sent false
17938 5771 Acked Read 10 Done Done Sent false
17939 5772 Acked Read 10 Done Done Sent false
17940 5773 Acked Write 1 Done Done Sent false
17941 5774 Acked Write 2 Done Done Sent false
17942 5775 Acked Read 2 Done Done Sent false
17943 5776 Acked Flush 0 Done Done Sent false
17944 5777 Acked Flush 0 Done Done Sent false
17945 5778 Acked Write 5 Done Done Sent false
17946 5779 Acked Write 9 Done Done Sent false
17947 5780 Acked Flush 0 Done Done Sent false
17948 5781 Acked Read 5 Done Done Sent false
17949 5782 Acked Flush 0 Done Done Sent false
17950 5783 Acked Write 3 Done Done Sent false
17951 5784 Acked Write 9 Done Done Sent false
17952 5785 Acked Flush 0 Done Done Sent false
17953 5786 Acked Read 7 Done Done Sent false
17954 5787 Acked Flush 0 Done Done Sent false
17955 5788 Acked Read 2 Done Done Sent false
17956 5789 Acked Write 3 Done Done Sent false
17957 5790 Acked Write 1 Done Done Sent false
17958 5791 Acked Write 2 Done Done Sent false
17959 5792 Acked Flush 0 Done Done Sent false
17960 5793 Acked Write 10 Done Done Sent false
17961 5794 Acked Flush 0 Done Done Sent false
17962 5795 Acked Read 6 Done Done Sent false
17963 5796 Acked Read 7 Done Done Sent false
17964 5797 Acked Read 8 Done Done Sent false
17965 5798 Acked Read 3 Done Done Sent false
17966 5799 Acked Write 8 Done Done Sent false
17967 5800 Acked Read 4 Done Done Sent false
17968 5801 Acked Read 2 Done Done Sent false
17969 5802 Acked Write 7 Done Done Sent false
17970 5803 Acked Write 10 Done Done Sent false
17971 5804 Acked Read 10 Done Done Sent false
17972 5805 Acked Write 9 Done Done Sent false
17973 5806 Acked Write 6 Done Done Sent false
17974 5807 Acked Read 4 Done Done Sent false
17975 5808 Acked Read 4 Done Done Sent false
17976 5809 Acked Write 9 Done Done Sent false
17977 5810 Acked Write 1 Done Done Sent false
17978 5811 Acked Read 3 Done Done Sent false
17979 5812 Acked Write 10 Done Done Sent false
17980 5813 Acked Write 7 Done Done Sent false
17981 5814 Acked Read 9 Done Done Sent false
17982 5815 Acked Write 10 Done Done Sent false
17983 5816 Acked Flush 0 Done Done Sent false
17984 5817 Acked Read 2 Done Done Sent false
17985 5818 Acked Read 3 Done Done Sent false
17986 5819 Acked Write 4 Done Done Sent false
17987 5820 Acked Read 2 Done Done Sent false
17988 5821 Acked Read 1 Done Done Sent false
17989 5822 Acked Read 6 Done Done Sent false
17990 5823 Acked Write 1 Done Done Sent false
17991 5824 Acked Write 4 Done Done Sent false
17992 5825 Acked Read 3 Done Done Sent false
17993 5826 Acked Read 5 Done Done Sent false
17994 5827 Acked Read 8 Done Done Sent false
17995 5828 Acked Write 10 Done Done Sent false
17996 5829 Acked Write 6 Done Done Sent false
17997 5830 Acked Write 5 Done Done Sent false
17998 5831 Acked Read 8 Done Done Sent false
17999 5832 Acked Flush 0 Done Done Sent false
18000 5833 Acked Flush 0 Done Done Sent false
18001 5834 Acked Read 1 Done Done Sent false
18002 5835 Acked Write 5 Done Done Sent false
18003 5836 Acked Read 2 Done Done Sent false
18004 STATES DS:0 DS:1 DS:2 TOTAL
18005 Sent 0 0 301 301
18006 Done 301 301 0 602
18007 Skipped 0 0 0 0
18008 Error 0 0 0 0
18009 Last Flush: 5833 5833 5535
18010 Downstairs last five completed: 5535 5534 5533 5532 5531
18011 Upstairs last five completed: 5836 5835 5834 5833 5832
18012 CLIENT: Up:0 ds:301 act:2
18013 ----------------------------------------------------------------
18014 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18015 States: Active Active Offline
18016 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18017 5536 Acked Read 5 Done Done Sent false
18018 5537 Acked Read 10 Done Done Sent false
18019 5538 Acked Read 2 Done Done Sent false
18020 5539 Acked Read 10 Done Done Sent false
18021 5540 Acked Read 5 Done Done Sent false
18022 5541 Acked Read 9 Done Done Sent false
18023 5542 Acked Read 2 Done Done Sent false
18024 5543 Acked Write 4 Done Done Sent false
18025 5544 Acked Read 4 Done Done Sent false
18026 5545 Acked Read 9 Done Done Sent false
18027 5546 Acked Write 8 Done Done Sent false
18028 5547 Acked Read 1 Done Done Sent false
18029 5548 Acked Read 8 Done Done Sent false
18030 5549 Acked Write 9 Done Done Sent false
18031 5550 Acked Read 9 Done Done Sent false
18032 5551 Acked Write 10 Done Done Sent false
18033 5552 Acked Write 4 Done Done Sent false
18034 5553 Acked Read 10 Done Done Sent false
18035 5554 Acked Write 5 Done Done Sent false
18036 5555 Acked Flush 0 Done Done Sent false
18037 5556 Acked Write 3 Done Done Sent false
18038 5557 Acked Write 3 Done Done Sent false
18039 5558 Acked Read 1 Done Done Sent false
18040 5559 Acked Write 2 Done Done Sent false
18041 5560 Acked Read 2 Done Done Sent false
18042 5561 Acked Read 8 Done Done Sent false
18043 5562 Acked Flush 0 Done Done Sent false
18044 5563 Acked Write 10 Done Done Sent false
18045 5564 Acked Read 1 Done Done Sent false
18046 5565 Acked Write 9 Done Done Sent false
18047 5566 Acked Flush 0 Done Done Sent false
18048 5567 Acked Read 5 Done Done Sent false
18049 5568 Acked Read 2 Done Done Sent false
18050 5569 Acked Read 6 Done Done Sent false
18051 5570 Acked Read 5 Done Done Sent false
18052 5571 Acked Read 3 Done Done Sent false
18053 5572 Acked Read 7 Done Done Sent false
18054 5573 Acked Read 4 Done Done Sent false
18055 5574 Acked Read 6 Done Done Sent false
18056 5575 Acked Read 5 Done Done Sent false
18057 5576 Acked Read 2 Done Done Sent false
18058 5577 Acked Flush 0 Done Done Sent false
18059 5578 Acked Write 5 Done Done Sent false
18060 5579 Acked Read 10 Done Done Sent false
18061 5580 Acked Write 10 Done Done Sent false
18062 5581 Acked Write 9 Done Done Sent false
18063 5582 Acked Write 9 Done Done Sent false
18064 5583 Acked Flush 0 Done Done Sent false
18065 5584 Acked Write 7 Done Done Sent false
18066 5585 Acked Read 3 Done Done Sent false
18067 5586 Acked Write 6 Done Done Sent false
18068 5587 Acked Write 4 Done Done Sent false
18069 5588 Acked Write 2 Done Done Sent false
18070 5589 Acked Read 5 Done Done Sent false
18071 5590 Acked Read 10 Done Done Sent false
18072 5591 Acked Write 3 Done Done Sent false
18073 5592 Acked Write 2 Done Done Sent false
18074 5593 Acked Write 3 Done Done Sent false
18075 5594 Acked Read 8 Done Done Sent false
18076 5595 Acked Write 10 Done Done Sent false
18077 5596 Acked Read 10 Done Done Sent false
18078 5597 Acked Write 8 Done Done Sent false
18079 5598 Acked Read 8 Done Done Sent false
18080 5599 Acked Read 9 Done Done Sent false
18081 5600 Acked Write 3 Done Done Sent false
18082 5601 Acked Read 5 Done Done Sent false
18083 5602 Acked Write 6 Done Done Sent false
18084 5603 Acked Read 7 Done Done Sent false
18085 5604 Acked Read 2 Done Done Sent false
18086 5605 Acked Write 10 Done Done Sent false
18087 5606 Acked Write 2 Done Done Sent false
18088 5607 Acked Read 4 Done Done Sent false
18089 5608 Acked Write 9 Done Done Sent false
18090 5609 Acked Flush 0 Done Done Sent false
18091 5610 Acked Write 6 Done Done Sent false
18092 5611 Acked Read 8 Done Done Sent false
18093 5612 Acked Read 2 Done Done Sent false
18094 5613 Acked Read 9 Done Done Sent false
18095 5614 Acked Read 3 Done Done Sent false
18096 5615 Acked Read 7 Done Done Sent false
18097 5616 Acked Write 3 Done Done Sent false
18098 5617 Acked Read 3 Done Done Sent false
18099 5618 Acked Read 6 Done Done Sent false
18100 5619 Acked Write 10 Done Done Sent false
18101 5620 Acked Read 5 Done Done Sent false
18102 5621 Acked Read 10 Done Done Sent false
18103 5622 Acked Write 9 Done Done Sent false
18104 5623 Acked Read 2 Done Done Sent false
18105 5624 Acked Read 7 Done Done Sent false
18106 5625 Acked Write 8 Done Done Sent false
18107 5626 Acked Write 7 Done Done Sent false
18108 5627 Acked Write 9 Done Done Sent false
18109 5628 Acked Read 9 Done Done Sent false
18110 5629 Acked Write 5 Done Done Sent false
18111 5630 Acked Read 10 Done Done Sent false
18112 5631 Acked Write 9 Done Done Sent false
18113 5632 Acked Write 2 Done Done Sent false
18114 5633 Acked Flush 0 Done Done Sent false
18115 5634 Acked Read 5 Done Done Sent false
18116 5635 Acked Read 2 Done Done Sent false
18117 5636 Acked Write 4 Done Done Sent false
18118 5637 Acked Read 6 Done Done Sent false
18119 5638 Acked Write 2 Done Done Sent false
18120 5639 Acked Read 9 Done Done Sent false
18121 5640 Acked Flush 0 Done Done Sent false
18122 5641 Acked Write 7 Done Done Sent false
18123 5642 Acked Read 3 Done Done Sent false
18124 5643 Acked Read 9 Done Done Sent false
18125 5644 Acked Read 6 Done Done Sent false
18126 5645 Acked Write 8 Done Done Sent false
18127 5646 Acked Write 4 Done Done Sent false
18128 5647 Acked Read 4 Done Done Sent false
18129 5648 Acked Flush 0 Done Done Sent false
18130 5649 Acked Read 3 Done Done Sent false
18131 5650 Acked Write 10 Done Done Sent false
18132 5651 Acked Read 8 Done Done Sent false
18133 5652 Acked Write 7 Done Done Sent false
18134 5653 Acked Write 6 Done Done Sent false
18135 5654 Acked Read 7 Done Done Sent false
18136 5655 Acked Flush 0 Done Done Sent false
18137 5656 Acked Read 7 Done Done Sent false
18138 5657 Acked Write 2 Done Done Sent false
18139 5658 Acked Write 10 Done Done Sent false
18140 5659 Acked Write 6 Done Done Sent false
18141 5660 Acked Write 10 Done Done Sent false
18142 5661 Acked Flush 0 Done Done Sent false
18143 5662 Acked Flush 0 Done Done Sent false
18144 5663 Acked Write 1 Done Done Sent false
18145 5664 Acked Read 4 Done Done Sent false
18146 5665 Acked Write 7 Done Done Sent false
18147 5666 Acked Write 2 Done Done Sent false
18148 5667 Acked Write 5 Done Done Sent false
18149 5668 Acked Write 7 Done Done Sent false
18150 5669 Acked Write 2 Done Done Sent false
18151 5670 Acked Read 2 Done Done Sent false
18152 5671 Acked Write 3 Done Done Sent false
18153 5672 Acked Write 1 Done Done Sent false
18154 5673 Acked Write 7 Done Done Sent false
18155 5674 Acked Write 10 Done Done Sent false
18156 5675 Acked Flush 0 Done Done Sent false
18157 5676 Acked Read 1 Done Done Sent false
18158 5677 Acked Read 6 Done Done Sent false
18159 5678 Acked Write 5 Done Done Sent false
18160 5679 Acked Write 8 Done Done Sent false
18161 5680 Acked Write 9 Done Done Sent false
18162 5681 Acked Read 5 Done Done Sent false
18163 5682 Acked Flush 0 Done Done Sent false
18164 5683 Acked Write 10 Done Done Sent false
18165 5684 Acked Read 3 Done Done Sent false
18166 5685 Acked Read 10 Done Done Sent false
18167 5686 Acked Write 4 Done Done Sent false
18168 5687 Acked Read 5 Done Done Sent false
18169 5688 Acked Flush 0 Done Done Sent false
18170 5689 Acked Write 9 Done Done Sent false
18171 5690 Acked Write 7 Done Done Sent false
18172 5691 Acked Read 10 Done Done Sent false
18173 5692 Acked Read 4 Done Done Sent false
18174 5693 Acked Read 8 Done Done Sent false
18175 5694 Acked Read 9 Done Done Sent false
18176 5695 Acked Write 4 Done Done Sent false
18177 5696 Acked Read 10 Done Done Sent false
18178 5697 Acked Write 5 Done Done Sent false
18179 5698 Acked Read 4 Done Done Sent false
18180 5699 Acked Read 10 Done Done Sent false
18181 5700 Acked Read 2 Done Done Sent false
18182 5701 Acked Write 3 Done Done Sent false
18183 5702 Acked Write 6 Done Done Sent false
18184 5703 Acked Read 6 Done Done Sent false
18185 5704 Acked Read 1 Done Done Sent false
18186 5705 Acked Read 1 Done Done Sent false
18187 5706 Acked Read 8 Done Done Sent false
18188 5707 Acked Read 4 Done Done Sent false
18189 5708 Acked Write 8 Done Done Sent false
18190 5709 Acked Read 4 Done Done Sent false
18191 5710 Acked Write 2 Done Done Sent false
18192 5711 Acked Read 6 Done Done Sent false
18193 5712 Acked Read 10 Done Done Sent false
18194 5713 Acked Flush 0 Done Done Sent false
18195 5714 Acked Read 6 Done Done Sent false
18196 5715 Acked Read 10 Done Done Sent false
18197 5716 Acked Flush 0 Done Done Sent false
18198 5717 Acked Read 10 Done Done Sent false
18199 5718 Acked Write 4 Done Done Sent false
18200 5719 Acked Flush 0 Done Done Sent false
18201 5720 Acked Read 9 Done Done Sent false
18202 5721 Acked Read 5 Done Done Sent false
18203 5722 Acked Write 5 Done Done Sent false
18204 5723 Acked Flush 0 Done Done Sent false
18205 5724 Acked Read 6 Done Done Sent false
18206 5725 Acked Read 9 Done Done Sent false
18207 5726 Acked Flush 0 Done Done Sent false
18208 5727 Acked Write 10 Done Done Sent false
18209 5728 Acked Write 8 Done Done Sent false
18210 5729 Acked Write 4 Done Done Sent false
18211 5730 Acked Read 9 Done Done Sent false
18212 5731 Acked Read 6 Done Done Sent false
18213 5732 Acked Read 3 Done Done Sent false
18214 5733 Acked Read 1 Done Done Sent false
18215 5734 Acked Read 8 Done Done Sent false
18216 5735 Acked Write 2 Done Done Sent false
18217 5736 Acked Flush 0 Done Done Sent false
18218 5737 Acked Read 3 Done Done Sent false
18219 5738 Acked Read 8 Done Done Sent false
18220 5739 Acked Read 10 Done Done Sent false
18221 5740 Acked Write 5 Done Done Sent false
18222 5741 Acked Write 7 Done Done Sent false
18223 5742 Acked Write 9 Done Done Sent false
18224 5743 Acked Write 4 Done Done Sent false
18225 5744 Acked Flush 0 Done Done Sent false
18226 5745 Acked Read 9 Done Done Sent false
18227 5746 Acked Write 2 Done Done Sent false
18228 5747 Acked Read 10 Done Done Sent false
18229 5748 Acked Read 7 Done Done Sent false
18230 5749 Acked Write 3 Done Done Sent false
18231 5750 Acked Write 5 Done Done Sent false
18232 5751 Acked Write 9 Done Done Sent false
18233 5752 Acked Read 3 Done Done Sent false
18234 5753 Acked Flush 0 Done Done Sent false
18235 5754 Acked Write 2 Done Done Sent false
18236 5755 Acked Read 3 Done Done Sent false
18237 5756 Acked Write 4 Done Done Sent false
18238 5757 Acked Read 8 Done Done Sent false
18239 5758 Acked Read 2 Done Done Sent false
18240 5759 Acked Write 2 Done Done Sent false
18241 5760 Acked Read 1 Done Done Sent false
18242 5761 Acked Read 3 Done Done Sent false
18243 5762 Acked Write 2 Done Done Sent false
18244 5763 Acked Write 3 Done Done Sent false
18245 5764 Acked Read 8 Done Done Sent false
18246 5765 Acked Read 1 Done Done Sent false
18247 5766 Acked Read 1 Done Done Sent false
18248 5767 Acked Read 10 Done Done Sent false
18249 5768 Acked Flush 0 Done Done Sent false
18250 5769 Acked Write 10 Done Done Sent false
18251 5770 Acked Read 3 Done Done Sent false
18252 5771 Acked Read 10 Done Done Sent false
18253 5772 Acked Read 10 Done Done Sent false
18254 5773 Acked Write 1 Done Done Sent false
18255 5774 Acked Write 2 Done Done Sent false
18256 5775 Acked Read 2 Done Done Sent false
18257 5776 Acked Flush 0 Done Done Sent false
18258 5777 Acked Flush 0 Done Done Sent false
18259 5778 Acked Write 5 Done Done Sent false
18260 5779 Acked Write 9 Done Done Sent false
18261 5780 Acked Flush 0 Done Done Sent false
18262 5781 Acked Read 5 Done Done Sent false
18263 5782 Acked Flush 0 Done Done Sent false
18264 5783 Acked Write 3 Done Done Sent false
18265 5784 Acked Write 9 Done Done Sent false
18266 5785 Acked Flush 0 Done Done Sent false
18267 5786 Acked Read 7 Done Done Sent false
18268 5787 Acked Flush 0 Done Done Sent false
18269 5788 Acked Read 2 Done Done Sent false
18270 5789 Acked Write 3 Done Done Sent false
18271 5790 Acked Write 1 Done Done Sent false
18272 5791 Acked Write 2 Done Done Sent false
18273 5792 Acked Flush 0 Done Done Sent false
18274 5793 Acked Write 10 Done Done Sent false
18275 5794 Acked Flush 0 Done Done Sent false
18276 5795 Acked Read 6 Done Done Sent false
18277 5796 Acked Read 7 Done Done Sent false
18278 5797 Acked Read 8 Done Done Sent false
18279 5798 Acked Read 3 Done Done Sent false
18280 5799 Acked Write 8 Done Done Sent false
18281 5800 Acked Read 4 Done Done Sent false
18282 5801 Acked Read 2 Done Done Sent false
18283 5802 Acked Write 7 Done Done Sent false
18284 5803 Acked Write 10 Done Done Sent false
18285 5804 Acked Read 10 Done Done Sent false
18286 5805 Acked Write 9 Done Done Sent false
18287 5806 Acked Write 6 Done Done Sent false
18288 5807 Acked Read 4 Done Done Sent false
18289 5808 Acked Read 4 Done Done Sent false
18290 5809 Acked Write 9 Done Done Sent false
18291 5810 Acked Write 1 Done Done Sent false
18292 5811 Acked Read 3 Done Done Sent false
18293 5812 Acked Write 10 Done Done Sent false
18294 5813 Acked Write 7 Done Done Sent false
18295 5814 Acked Read 9 Done Done Sent false
18296 5815 Acked Write 10 Done Done Sent false
18297 5816 Acked Flush 0 Done Done Sent false
18298 5817 Acked Read 2 Done Done Sent false
18299 5818 Acked Read 3 Done Done Sent false
18300 5819 Acked Write 4 Done Done Sent false
18301 5820 Acked Read 2 Done Done Sent false
18302 5821 Acked Read 1 Done Done Sent false
18303 5822 Acked Read 6 Done Done Sent false
18304 5823 Acked Write 1 Done Done Sent false
18305 5824 Acked Write 4 Done Done Sent false
18306 5825 Acked Read 3 Done Done Sent false
18307 5826 Acked Read 5 Done Done Sent false
18308 5827 Acked Read 8 Done Done Sent false
18309 5828 Acked Write 10 Done Done Sent false
18310 5829 Acked Write 6 Done Done Sent false
18311 5830 Acked Write 5 Done Done Sent false
18312 5831 Acked Read 8 Done Done Sent false
18313 5832 Acked Flush 0 Done Done Sent false
18314 5833 Acked Flush 0 Done Done Sent false
18315 5834 Acked Read 1 Done Done Sent false
18316 5835 Acked Write 5 Done Done Sent false
18317 5836 Acked Read 2 Done Done Sent false
18318 5837 Acked Flush 0 Done Done Sent false
18319 STATES DS:0 DS:1 DS:2 TOTAL
18320 Sent 0 0 302 302
18321 Done 302 302 0 604
18322 Skipped 0 0 0 0
18323 Error 0 0 0 0
18324 Last Flush: 5837 5837 5535
18325 Downstairs last five completed: 5535 5534 5533 5532 5531
18326 Upstairs last five completed: 5837 5836 5835 5834 5833
18327 CLIENT: Up:0 ds:302 act:2
183282024-11-07T14:07:08.259ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
183292024-11-07T14:07:08.259ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
183302024-11-07T14:07:08.259ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
183312024-11-07T14:07:08.259ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
183322024-11-07T14:07:08.259ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
18333 ----------------------------------------------------------------
18334 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18335 States: Active Active Offline
18336 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18337 5536 Acked Read 5 Done Done Sent false
18338 5537 Acked Read 10 Done Done Sent false
18339 5538 Acked Read 2 Done Done Sent false
18340 5539 Acked Read 10 Done Done Sent false
18341 5540 Acked Read 5 Done Done Sent false
18342 5541 Acked Read 9 Done Done Sent false
18343 5542 Acked Read 2 Done Done Sent false
18344 5543 Acked Write 4 Done Done Sent false
18345 5544 Acked Read 4 Done Done Sent false
18346 5545 Acked Read 9 Done Done Sent false
18347 5546 Acked Write 8 Done Done Sent false
18348 5547 Acked Read 1 Done Done Sent false
18349 5548 Acked Read 8 Done Done Sent false
18350 5549 Acked Write 9 Done Done Sent false
18351 5550 Acked Read 9 Done Done Sent false
18352 5551 Acked Write 10 Done Done Sent false
18353 5552 Acked Write 4 Done Done Sent false
18354 5553 Acked Read 10 Done Done Sent false
18355 5554 Acked Write 5 Done Done Sent false
18356 5555 Acked Flush 0 Done Done Sent false
18357 5556 Acked Write 3 Done Done Sent false
18358 5557 Acked Write 3 Done Done Sent false
18359 5558 Acked Read 1 Done Done Sent false
18360 5559 Acked Write 2 Done Done Sent false
18361 5560 Acked Read 2 Done Done Sent false
18362 5561 Acked Read 8 Done Done Sent false
18363 5562 Acked Flush 0 Done Done Sent false
18364 5563 Acked Write 10 Done Done Sent false
18365 5564 Acked Read 1 Done Done Sent false
18366 5565 Acked Write 9 Done Done Sent false
18367 5566 Acked Flush 0 Done Done Sent false
18368 5567 Acked Read 5 Done Done Sent false
18369 5568 Acked Read 2 Done Done Sent false
18370 5569 Acked Read 6 Done Done Sent false
18371 5570 Acked Read 5 Done Done Sent false
18372 5571 Acked Read 3 Done Done Sent false
18373 5572 Acked Read 7 Done Done Sent false
18374 5573 Acked Read 4 Done Done Sent false
18375 5574 Acked Read 6 Done Done Sent false
18376 5575 Acked Read 5 Done Done Sent false
18377 5576 Acked Read 2 Done Done Sent false
18378 5577 Acked Flush 0 Done Done Sent false
18379 5578 Acked Write 5 Done Done Sent false
18380 5579 Acked Read 10 Done Done Sent false
18381 5580 Acked Write 10 Done Done Sent false
18382 5581 Acked Write 9 Done Done Sent false
18383 5582 Acked Write 9 Done Done Sent false
18384 5583 Acked Flush 0 Done Done Sent false
18385 5584 Acked Write 7 Done Done Sent false
18386 5585 Acked Read 3 Done Done Sent false
18387 5586 Acked Write 6 Done Done Sent false
18388 5587 Acked Write 4 Done Done Sent false
18389 5588 Acked Write 2 Done Done Sent false
18390 5589 Acked Read 5 Done Done Sent false
18391 5590 Acked Read 10 Done Done Sent false
18392 5591 Acked Write 3 Done Done Sent false
18393 5592 Acked Write 2 Done Done Sent false
18394 5593 Acked Write 3 Done Done Sent false
18395 5594 Acked Read 8 Done Done Sent false
18396 5595 Acked Write 10 Done Done Sent false
18397 5596 Acked Read 10 Done Done Sent false
18398 5597 Acked Write 8 Done Done Sent false
18399 5598 Acked Read 8 Done Done Sent false
18400 5599 Acked Read 9 Done Done Sent false
18401 5600 Acked Write 3 Done Done Sent false
18402 5601 Acked Read 5 Done Done Sent false
18403 5602 Acked Write 6 Done Done Sent false
18404 5603 Acked Read 7 Done Done Sent false
18405 5604 Acked Read 2 Done Done Sent false
18406 5605 Acked Write 10 Done Done Sent false
18407 5606 Acked Write 2 Done Done Sent false
18408 5607 Acked Read 4 Done Done Sent false
18409 5608 Acked Write 9 Done Done Sent false
18410 5609 Acked Flush 0 Done Done Sent false
18411 5610 Acked Write 6 Done Done Sent false
18412 5611 Acked Read 8 Done Done Sent false
18413 5612 Acked Read 2 Done Done Sent false
18414 5613 Acked Read 9 Done Done Sent false
18415 5614 Acked Read 3 Done Done Sent false
18416 5615 Acked Read 7 Done Done Sent false
18417 5616 Acked Write 3 Done Done Sent false
18418 5617 Acked Read 3 Done Done Sent false
18419 5618 Acked Read 6 Done Done Sent false
18420 5619 Acked Write 10 Done Done Sent false
18421 5620 Acked Read 5 Done Done Sent false
18422 5621 Acked Read 10 Done Done Sent false
18423 5622 Acked Write 9 Done Done Sent false
18424 5623 Acked Read 2 Done Done Sent false
18425 5624 Acked Read 7 Done Done Sent false
18426 5625 Acked Write 8 Done Done Sent false
18427 5626 Acked Write 7 Done Done Sent false
18428 5627 Acked Write 9 Done Done Sent false
18429 5628 Acked Read 9 Done Done Sent false
18430 5629 Acked Write 5 Done Done Sent false
18431 5630 Acked Read 10 Done Done Sent false
18432 5631 Acked Write 9 Done Done Sent false
18433 5632 Acked Write 2 Done Done Sent false
18434 5633 Acked Flush 0 Done Done Sent false
18435 5634 Acked Read 5 Done Done Sent false
18436 5635 Acked Read 2 Done Done Sent false
18437 5636 Acked Write 4 Done Done Sent false
18438 5637 Acked Read 6 Done Done Sent false
18439 5638 Acked Write 2 Done Done Sent false
18440 5639 Acked Read 9 Done Done Sent false
18441 5640 Acked Flush 0 Done Done Sent false
18442 5641 Acked Write 7 Done Done Sent false
18443 5642 Acked Read 3 Done Done Sent false
18444 5643 Acked Read 9 Done Done Sent false
18445 5644 Acked Read 6 Done Done Sent false
18446 5645 Acked Write 8 Done Done Sent false
18447 5646 Acked Write 4 Done Done Sent false
18448 5647 Acked Read 4 Done Done Sent false
18449 5648 Acked Flush 0 Done Done Sent false
18450 5649 Acked Read 3 Done Done Sent false
18451 5650 Acked Write 10 Done Done Sent false
18452 5651 Acked Read 8 Done Done Sent false
18453 5652 Acked Write 7 Done Done Sent false
18454 5653 Acked Write 6 Done Done Sent false
18455 5654 Acked Read 7 Done Done Sent false
18456 5655 Acked Flush 0 Done Done Sent false
18457 5656 Acked Read 7 Done Done Sent false
18458 5657 Acked Write 2 Done Done Sent false
18459 5658 Acked Write 10 Done Done Sent false
18460 5659 Acked Write 6 Done Done Sent false
18461 5660 Acked Write 10 Done Done Sent false
18462 5661 Acked Flush 0 Done Done Sent false
18463 5662 Acked Flush 0 Done Done Sent false
18464 5663 Acked Write 1 Done Done Sent false
18465 5664 Acked Read 4 Done Done Sent false
18466 5665 Acked Write 7 Done Done Sent false
18467 5666 Acked Write 2 Done Done Sent false
18468 5667 Acked Write 5 Done Done Sent false
18469 5668 Acked Write 7 Done Done Sent false
18470 5669 Acked Write 2 Done Done Sent false
18471 5670 Acked Read 2 Done Done Sent false
18472 5671 Acked Write 3 Done Done Sent false
18473 5672 Acked Write 1 Done Done Sent false
18474 5673 Acked Write 7 Done Done Sent false
18475 5674 Acked Write 10 Done Done Sent false
18476 5675 Acked Flush 0 Done Done Sent false
18477 5676 Acked Read 1 Done Done Sent false
18478 5677 Acked Read 6 Done Done Sent false
18479 5678 Acked Write 5 Done Done Sent false
18480 5679 Acked Write 8 Done Done Sent false
18481 5680 Acked Write 9 Done Done Sent false
18482 5681 Acked Read 5 Done Done Sent false
18483 5682 Acked Flush 0 Done Done Sent false
18484 5683 Acked Write 10 Done Done Sent false
18485 5684 Acked Read 3 Done Done Sent false
18486 5685 Acked Read 10 Done Done Sent false
18487 5686 Acked Write 4 Done Done Sent false
18488 5687 Acked Read 5 Done Done Sent false
18489 5688 Acked Flush 0 Done Done Sent false
18490 5689 Acked Write 9 Done Done Sent false
18491 5690 Acked Write 7 Done Done Sent false
18492 5691 Acked Read 10 Done Done Sent false
18493 5692 Acked Read 4 Done Done Sent false
18494 5693 Acked Read 8 Done Done Sent false
18495 5694 Acked Read 9 Done Done Sent false
18496 5695 Acked Write 4 Done Done Sent false
18497 5696 Acked Read 10 Done Done Sent false
18498 5697 Acked Write 5 Done Done Sent false
18499 5698 Acked Read 4 Done Done Sent false
18500 5699 Acked Read 10 Done Done Sent false
18501 5700 Acked Read 2 Done Done Sent false
18502 5701 Acked Write 3 Done Done Sent false
18503 5702 Acked Write 6 Done Done Sent false
18504 5703 Acked Read 6 Done Done Sent false
18505 5704 Acked Read 1 Done Done Sent false
18506 5705 Acked Read 1 Done Done Sent false
18507 5706 Acked Read 8 Done Done Sent false
18508 5707 Acked Read 4 Done Done Sent false
18509 5708 Acked Write 8 Done Done Sent false
18510 5709 Acked Read 4 Done Done Sent false
18511 5710 Acked Write 2 Done Done Sent false
18512 5711 Acked Read 6 Done Done Sent false
18513 5712 Acked Read 10 Done Done Sent false
18514 5713 Acked Flush 0 Done Done Sent false
18515 5714 Acked Read 6 Done Done Sent false
18516 5715 Acked Read 10 Done Done Sent false
18517 5716 Acked Flush 0 Done Done Sent false
18518 5717 Acked Read 10 Done Done Sent false
18519 5718 Acked Write 4 Done Done Sent false
18520 5719 Acked Flush 0 Done Done Sent false
18521 5720 Acked Read 9 Done Done Sent false
18522 5721 Acked Read 5 Done Done Sent false
18523 5722 Acked Write 5 Done Done Sent false
18524 5723 Acked Flush 0 Done Done Sent false
18525 5724 Acked Read 6 Done Done Sent false
18526 5725 Acked Read 9 Done Done Sent false
18527 5726 Acked Flush 0 Done Done Sent false
18528 5727 Acked Write 10 Done Done Sent false
18529 5728 Acked Write 8 Done Done Sent false
18530 5729 Acked Write 4 Done Done Sent false
18531 5730 Acked Read 9 Done Done Sent false
18532 5731 Acked Read 6 Done Done Sent false
18533 5732 Acked Read 3 Done Done Sent false
18534 5733 Acked Read 1 Done Done Sent false
18535 5734 Acked Read 8 Done Done Sent false
18536 5735 Acked Write 2 Done Done Sent false
18537 5736 Acked Flush 0 Done Done Sent false
18538 5737 Acked Read 3 Done Done Sent false
18539 5738 Acked Read 8 Done Done Sent false
18540 5739 Acked Read 10 Done Done Sent false
18541 5740 Acked Write 5 Done Done Sent false
18542 5741 Acked Write 7 Done Done Sent false
18543 5742 Acked Write 9 Done Done Sent false
18544 5743 Acked Write 4 Done Done Sent false
18545 5744 Acked Flush 0 Done Done Sent false
18546 5745 Acked Read 9 Done Done Sent false
18547 5746 Acked Write 2 Done Done Sent false
18548 5747 Acked Read 10 Done Done Sent false
18549 5748 Acked Read 7 Done Done Sent false
18550 5749 Acked Write 3 Done Done Sent false
18551 5750 Acked Write 5 Done Done Sent false
18552 5751 Acked Write 9 Done Done Sent false
18553 5752 Acked Read 3 Done Done Sent false
18554 5753 Acked Flush 0 Done Done Sent false
18555 5754 Acked Write 2 Done Done Sent false
18556 5755 Acked Read 3 Done Done Sent false
18557 5756 Acked Write 4 Done Done Sent false
18558 5757 Acked Read 8 Done Done Sent false
18559 5758 Acked Read 2 Done Done Sent false
18560 5759 Acked Write 2 Done Done Sent false
18561 5760 Acked Read 1 Done Done Sent false
18562 5761 Acked Read 3 Done Done Sent false
18563 5762 Acked Write 2 Done Done Sent false
18564 5763 Acked Write 3 Done Done Sent false
18565 5764 Acked Read 8 Done Done Sent false
18566 5765 Acked Read 1 Done Done Sent false
18567 5766 Acked Read 1 Done Done Sent false
18568 5767 Acked Read 10 Done Done Sent false
18569 5768 Acked Flush 0 Done Done Sent false
18570 5769 Acked Write 10 Done Done Sent false
18571 5770 Acked Read 3 Done Done Sent false
18572 5771 Acked Read 10 Done Done Sent false
18573 5772 Acked Read 10 Done Done Sent false
18574 5773 Acked Write 1 Done Done Sent false
18575 5774 Acked Write 2 Done Done Sent false
18576 5775 Acked Read 2 Done Done Sent false
18577 5776 Acked Flush 0 Done Done Sent false
18578 5777 Acked Flush 0 Done Done Sent false
18579 5778 Acked Write 5 Done Done Sent false
18580 5779 Acked Write 9 Done Done Sent false
18581 5780 Acked Flush 0 Done Done Sent false
18582 5781 Acked Read 5 Done Done Sent false
18583 5782 Acked Flush 0 Done Done Sent false
18584 5783 Acked Write 3 Done Done Sent false
18585 5784 Acked Write 9 Done Done Sent false
18586 5785 Acked Flush 0 Done Done Sent false
18587 5786 Acked Read 7 Done Done Sent false
18588 5787 Acked Flush 0 Done Done Sent false
18589 5788 Acked Read 2 Done Done Sent false
18590 5789 Acked Write 3 Done Done Sent false
18591 5790 Acked Write 1 Done Done Sent false
18592 5791 Acked Write 2 Done Done Sent false
18593 5792 Acked Flush 0 Done Done Sent false
18594 5793 Acked Write 10 Done Done Sent false
18595 5794 Acked Flush 0 Done Done Sent false
18596 5795 Acked Read 6 Done Done Sent false
18597 5796 Acked Read 7 Done Done Sent false
18598 5797 Acked Read 8 Done Done Sent false
18599 5798 Acked Read 3 Done Done Sent false
18600 5799 Acked Write 8 Done Done Sent false
18601 5800 Acked Read 4 Done Done Sent false
18602 5801 Acked Read 2 Done Done Sent false
18603 5802 Acked Write 7 Done Done Sent false
18604 5803 Acked Write 10 Done Done Sent false
18605 5804 Acked Read 10 Done Done Sent false
18606 5805 Acked Write 9 Done Done Sent false
18607 5806 Acked Write 6 Done Done Sent false
18608 5807 Acked Read 4 Done Done Sent false
18609 5808 Acked Read 4 Done Done Sent false
18610 5809 Acked Write 9 Done Done Sent false
18611 5810 Acked Write 1 Done Done Sent false
18612 5811 Acked Read 3 Done Done Sent false
18613 5812 Acked Write 10 Done Done Sent false
18614 5813 Acked Write 7 Done Done Sent false
18615 5814 Acked Read 9 Done Done Sent false
18616 5815 Acked Write 10 Done Done Sent false
18617 5816 Acked Flush 0 Done Done Sent false
18618 5817 Acked Read 2 Done Done Sent false
18619 5818 Acked Read 3 Done Done Sent false
18620 5819 Acked Write 4 Done Done Sent false
18621 5820 Acked Read 2 Done Done Sent false
18622 5821 Acked Read 1 Done Done Sent false
18623 5822 Acked Read 6 Done Done Sent false
18624 5823 Acked Write 1 Done Done Sent false
18625 5824 Acked Write 4 Done Done Sent false
18626 5825 Acked Read 3 Done Done Sent false
18627 5826 Acked Read 5 Done Done Sent false
18628 5827 Acked Read 8 Done Done Sent false
18629 5828 Acked Write 10 Done Done Sent false
18630 5829 Acked Write 6 Done Done Sent false
18631 5830 Acked Write 5 Done Done Sent false
18632 5831 Acked Read 8 Done Done Sent false
18633 5832 Acked Flush 0 Done Done Sent false
18634 5833 Acked Flush 0 Done Done Sent false
18635 5834 Acked Read 1 Done Done Sent false
18636 5835 Acked Write 5 Done Done Sent false
18637 5836 Acked Read 2 Done Done Sent false
18638 5837 Acked Flush 0 Done Done Sent false
18639 STATES DS:0 DS:1 DS:2 TOTAL
18640 Sent 0 0 302 302
18641 Done 302 302 0 604
18642 Skipped 0 0 0 0
18643 Error 0 0 0 0
18644 Last Flush: 5837 5837 5535
18645 Downstairs last five completed: 5535 5534 5533 5532 5531
18646 Upstairs last five completed: 5837 5836 5835 5834 5833
18647 CLIENT: Up:0 ds:302 act:2
18648 ----------------------------------------------------------------
18649 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18650 States: Active Active Offline
18651 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18652 5536 Acked Read 5 Done Done Sent false
18653 5537 Acked Read 10 Done Done Sent false
18654 5538 Acked Read 2 Done Done Sent false
18655 5539 Acked Read 10 Done Done Sent false
18656 5540 Acked Read 5 Done Done Sent false
18657 5541 Acked Read 9 Done Done Sent false
18658 5542 Acked Read 2 Done Done Sent false
18659 5543 Acked Write 4 Done Done Sent false
18660 5544 Acked Read 4 Done Done Sent false
18661 5545 Acked Read 9 Done Done Sent false
18662 5546 Acked Write 8 Done Done Sent false
18663 5547 Acked Read 1 Done Done Sent false
18664 5548 Acked Read 8 Done Done Sent false
18665 5549 Acked Write 9 Done Done Sent false
18666 5550 Acked Read 9 Done Done Sent false
18667 5551 Acked Write 10 Done Done Sent false
18668 5552 Acked Write 4 Done Done Sent false
18669 5553 Acked Read 10 Done Done Sent false
18670 5554 Acked Write 5 Done Done Sent false
18671 5555 Acked Flush 0 Done Done Sent false
18672 5556 Acked Write 3 Done Done Sent false
18673 5557 Acked Write 3 Done Done Sent false
18674 5558 Acked Read 1 Done Done Sent false
18675 5559 Acked Write 2 Done Done Sent false
18676 5560 Acked Read 2 Done Done Sent false
18677 5561 Acked Read 8 Done Done Sent false
18678 5562 Acked Flush 0 Done Done Sent false
18679 5563 Acked Write 10 Done Done Sent false
18680 5564 Acked Read 1 Done Done Sent false
18681 5565 Acked Write 9 Done Done Sent false
18682 5566 Acked Flush 0 Done Done Sent false
18683 5567 Acked Read 5 Done Done Sent false
18684 5568 Acked Read 2 Done Done Sent false
18685 5569 Acked Read 6 Done Done Sent false
18686 5570 Acked Read 5 Done Done Sent false
18687 5571 Acked Read 3 Done Done Sent false
18688 5572 Acked Read 7 Done Done Sent false
18689 5573 Acked Read 4 Done Done Sent false
18690 5574 Acked Read 6 Done Done Sent false
18691 5575 Acked Read 5 Done Done Sent false
18692 5576 Acked Read 2 Done Done Sent false
18693 5577 Acked Flush 0 Done Done Sent false
18694 5578 Acked Write 5 Done Done Sent false
18695 5579 Acked Read 10 Done Done Sent false
18696 5580 Acked Write 10 Done Done Sent false
18697 5581 Acked Write 9 Done Done Sent false
18698 5582 Acked Write 9 Done Done Sent false
18699 5583 Acked Flush 0 Done Done Sent false
18700 5584 Acked Write 7 Done Done Sent false
18701 5585 Acked Read 3 Done Done Sent false
18702 5586 Acked Write 6 Done Done Sent false
18703 5587 Acked Write 4 Done Done Sent false
18704 5588 Acked Write 2 Done Done Sent false
18705 5589 Acked Read 5 Done Done Sent false
18706 5590 Acked Read 10 Done Done Sent false
18707 5591 Acked Write 3 Done Done Sent false
18708 5592 Acked Write 2 Done Done Sent false
18709 5593 Acked Write 3 Done Done Sent false
18710 5594 Acked Read 8 Done Done Sent false
18711 5595 Acked Write 10 Done Done Sent false
18712 5596 Acked Read 10 Done Done Sent false
18713 5597 Acked Write 8 Done Done Sent false
18714 5598 Acked Read 8 Done Done Sent false
18715 5599 Acked Read 9 Done Done Sent false
18716 5600 Acked Write 3 Done Done Sent false
18717 5601 Acked Read 5 Done Done Sent false
18718 5602 Acked Write 6 Done Done Sent false
18719 5603 Acked Read 7 Done Done Sent false
18720 5604 Acked Read 2 Done Done Sent false
18721 5605 Acked Write 10 Done Done Sent false
18722 5606 Acked Write 2 Done Done Sent false
18723 5607 Acked Read 4 Done Done Sent false
18724 5608 Acked Write 9 Done Done Sent false
18725 5609 Acked Flush 0 Done Done Sent false
18726 5610 Acked Write 6 Done Done Sent false
18727 5611 Acked Read 8 Done Done Sent false
18728 5612 Acked Read 2 Done Done Sent false
18729 5613 Acked Read 9 Done Done Sent false
18730 5614 Acked Read 3 Done Done Sent false
18731 5615 Acked Read 7 Done Done Sent false
18732 5616 Acked Write 3 Done Done Sent false
18733 5617 Acked Read 3 Done Done Sent false
18734 5618 Acked Read 6 Done Done Sent false
18735 5619 Acked Write 10 Done Done Sent false
18736 5620 Acked Read 5 Done Done Sent false
18737 5621 Acked Read 10 Done Done Sent false
18738 5622 Acked Write 9 Done Done Sent false
18739 5623 Acked Read 2 Done Done Sent false
18740 5624 Acked Read 7 Done Done Sent false
18741 5625 Acked Write 8 Done Done Sent false
18742 5626 Acked Write 7 Done Done Sent false
18743 5627 Acked Write 9 Done Done Sent false
18744 5628 Acked Read 9 Done Done Sent false
18745 5629 Acked Write 5 Done Done Sent false
18746 5630 Acked Read 10 Done Done Sent false
18747 5631 Acked Write 9 Done Done Sent false
18748 5632 Acked Write 2 Done Done Sent false
18749 5633 Acked Flush 0 Done Done Sent false
18750 5634 Acked Read 5 Done Done Sent false
18751 5635 Acked Read 2 Done Done Sent false
18752 5636 Acked Write 4 Done Done Sent false
18753 5637 Acked Read 6 Done Done Sent false
18754 5638 Acked Write 2 Done Done Sent false
18755 5639 Acked Read 9 Done Done Sent false
18756 5640 Acked Flush 0 Done Done Sent false
18757 5641 Acked Write 7 Done Done Sent false
18758 5642 Acked Read 3 Done Done Sent false
18759 5643 Acked Read 9 Done Done Sent false
18760 5644 Acked Read 6 Done Done Sent false
18761 5645 Acked Write 8 Done Done Sent false
18762 5646 Acked Write 4 Done Done Sent false
18763 5647 Acked Read 4 Done Done Sent false
18764 5648 Acked Flush 0 Done Done Sent false
18765 5649 Acked Read 3 Done Done Sent false
18766 5650 Acked Write 10 Done Done Sent false
18767 5651 Acked Read 8 Done Done Sent false
18768 5652 Acked Write 7 Done Done Sent false
18769 5653 Acked Write 6 Done Done Sent false
18770 5654 Acked Read 7 Done Done Sent false
18771 5655 Acked Flush 0 Done Done Sent false
18772 5656 Acked Read 7 Done Done Sent false
18773 5657 Acked Write 2 Done Done Sent false
18774 5658 Acked Write 10 Done Done Sent false
18775 5659 Acked Write 6 Done Done Sent false
18776 5660 Acked Write 10 Done Done Sent false
18777 5661 Acked Flush 0 Done Done Sent false
18778 5662 Acked Flush 0 Done Done Sent false
18779 5663 Acked Write 1 Done Done Sent false
18780 5664 Acked Read 4 Done Done Sent false
18781 5665 Acked Write 7 Done Done Sent false
18782 5666 Acked Write 2 Done Done Sent false
18783 5667 Acked Write 5 Done Done Sent false
18784 5668 Acked Write 7 Done Done Sent false
18785 5669 Acked Write 2 Done Done Sent false
18786 5670 Acked Read 2 Done Done Sent false
18787 5671 Acked Write 3 Done Done Sent false
18788 5672 Acked Write 1 Done Done Sent false
18789 5673 Acked Write 7 Done Done Sent false
18790 5674 Acked Write 10 Done Done Sent false
18791 5675 Acked Flush 0 Done Done Sent false
18792 5676 Acked Read 1 Done Done Sent false
18793 5677 Acked Read 6 Done Done Sent false
18794 5678 Acked Write 5 Done Done Sent false
18795 5679 Acked Write 8 Done Done Sent false
18796 5680 Acked Write 9 Done Done Sent false
18797 5681 Acked Read 5 Done Done Sent false
18798 5682 Acked Flush 0 Done Done Sent false
18799 5683 Acked Write 10 Done Done Sent false
18800 5684 Acked Read 3 Done Done Sent false
18801 5685 Acked Read 10 Done Done Sent false
18802 5686 Acked Write 4 Done Done Sent false
18803 5687 Acked Read 5 Done Done Sent false
18804 5688 Acked Flush 0 Done Done Sent false
18805 5689 Acked Write 9 Done Done Sent false
18806 5690 Acked Write 7 Done Done Sent false
18807 5691 Acked Read 10 Done Done Sent false
18808 5692 Acked Read 4 Done Done Sent false
18809 5693 Acked Read 8 Done Done Sent false
18810 5694 Acked Read 9 Done Done Sent false
18811 5695 Acked Write 4 Done Done Sent false
18812 5696 Acked Read 10 Done Done Sent false
18813 5697 Acked Write 5 Done Done Sent false
18814 5698 Acked Read 4 Done Done Sent false
18815 5699 Acked Read 10 Done Done Sent false
18816 5700 Acked Read 2 Done Done Sent false
18817 5701 Acked Write 3 Done Done Sent false
18818 5702 Acked Write 6 Done Done Sent false
18819 5703 Acked Read 6 Done Done Sent false
18820 5704 Acked Read 1 Done Done Sent false
18821 5705 Acked Read 1 Done Done Sent false
18822 5706 Acked Read 8 Done Done Sent false
18823 5707 Acked Read 4 Done Done Sent false
18824 5708 Acked Write 8 Done Done Sent false
18825 5709 Acked Read 4 Done Done Sent false
18826 5710 Acked Write 2 Done Done Sent false
18827 5711 Acked Read 6 Done Done Sent false
18828 5712 Acked Read 10 Done Done Sent false
18829 5713 Acked Flush 0 Done Done Sent false
18830 5714 Acked Read 6 Done Done Sent false
18831 5715 Acked Read 10 Done Done Sent false
18832 5716 Acked Flush 0 Done Done Sent false
18833 5717 Acked Read 10 Done Done Sent false
18834 5718 Acked Write 4 Done Done Sent false
18835 5719 Acked Flush 0 Done Done Sent false
18836 5720 Acked Read 9 Done Done Sent false
18837 5721 Acked Read 5 Done Done Sent false
18838 5722 Acked Write 5 Done Done Sent false
18839 5723 Acked Flush 0 Done Done Sent false
18840 5724 Acked Read 6 Done Done Sent false
18841 5725 Acked Read 9 Done Done Sent false
18842 5726 Acked Flush 0 Done Done Sent false
18843 5727 Acked Write 10 Done Done Sent false
18844 5728 Acked Write 8 Done Done Sent false
18845 5729 Acked Write 4 Done Done Sent false
18846 5730 Acked Read 9 Done Done Sent false
18847 5731 Acked Read 6 Done Done Sent false
18848 5732 Acked Read 3 Done Done Sent false
18849 5733 Acked Read 1 Done Done Sent false
18850 5734 Acked Read 8 Done Done Sent false
18851 5735 Acked Write 2 Done Done Sent false
18852 5736 Acked Flush 0 Done Done Sent false
18853 5737 Acked Read 3 Done Done Sent false
18854 5738 Acked Read 8 Done Done Sent false
18855 5739 Acked Read 10 Done Done Sent false
18856 5740 Acked Write 5 Done Done Sent false
18857 5741 Acked Write 7 Done Done Sent false
18858 5742 Acked Write 9 Done Done Sent false
18859 5743 Acked Write 4 Done Done Sent false
18860 5744 Acked Flush 0 Done Done Sent false
18861 5745 Acked Read 9 Done Done Sent false
18862 5746 Acked Write 2 Done Done Sent false
18863 5747 Acked Read 10 Done Done Sent false
18864 5748 Acked Read 7 Done Done Sent false
18865 5749 Acked Write 3 Done Done Sent false
18866 5750 Acked Write 5 Done Done Sent false
18867 5751 Acked Write 9 Done Done Sent false
18868 5752 Acked Read 3 Done Done Sent false
18869 5753 Acked Flush 0 Done Done Sent false
18870 5754 Acked Write 2 Done Done Sent false
18871 5755 Acked Read 3 Done Done Sent false
18872 5756 Acked Write 4 Done Done Sent false
18873 5757 Acked Read 8 Done Done Sent false
18874 5758 Acked Read 2 Done Done Sent false
18875 5759 Acked Write 2 Done Done Sent false
18876 5760 Acked Read 1 Done Done Sent false
18877 5761 Acked Read 3 Done Done Sent false
18878 5762 Acked Write 2 Done Done Sent false
18879 5763 Acked Write 3 Done Done Sent false
18880 5764 Acked Read 8 Done Done Sent false
18881 5765 Acked Read 1 Done Done Sent false
18882 5766 Acked Read 1 Done Done Sent false
18883 5767 Acked Read 10 Done Done Sent false
18884 5768 Acked Flush 0 Done Done Sent false
18885 5769 Acked Write 10 Done Done Sent false
18886 5770 Acked Read 3 Done Done Sent false
18887 5771 Acked Read 10 Done Done Sent false
18888 5772 Acked Read 10 Done Done Sent false
18889 5773 Acked Write 1 Done Done Sent false
18890 5774 Acked Write 2 Done Done Sent false
18891 5775 Acked Read 2 Done Done Sent false
18892 5776 Acked Flush 0 Done Done Sent false
18893 5777 Acked Flush 0 Done Done Sent false
18894 5778 Acked Write 5 Done Done Sent false
18895 5779 Acked Write 9 Done Done Sent false
18896 5780 Acked Flush 0 Done Done Sent false
18897 5781 Acked Read 5 Done Done Sent false
18898 5782 Acked Flush 0 Done Done Sent false
18899 5783 Acked Write 3 Done Done Sent false
18900 5784 Acked Write 9 Done Done Sent false
18901 5785 Acked Flush 0 Done Done Sent false
18902 5786 Acked Read 7 Done Done Sent false
18903 5787 Acked Flush 0 Done Done Sent false
18904 5788 Acked Read 2 Done Done Sent false
18905 5789 Acked Write 3 Done Done Sent false
18906 5790 Acked Write 1 Done Done Sent false
18907 5791 Acked Write 2 Done Done Sent false
18908 5792 Acked Flush 0 Done Done Sent false
18909 5793 Acked Write 10 Done Done Sent false
18910 5794 Acked Flush 0 Done Done Sent false
18911 5795 Acked Read 6 Done Done Sent false
18912 5796 Acked Read 7 Done Done Sent false
18913 5797 Acked Read 8 Done Done Sent false
18914 5798 Acked Read 3 Done Done Sent false
18915 5799 Acked Write 8 Done Done Sent false
18916 5800 Acked Read 4 Done Done Sent false
18917 5801 Acked Read 2 Done Done Sent false
18918 5802 Acked Write 7 Done Done Sent false
18919 5803 Acked Write 10 Done Done Sent false
18920 5804 Acked Read 10 Done Done Sent false
18921 5805 Acked Write 9 Done Done Sent false
18922 5806 Acked Write 6 Done Done Sent false
18923 5807 Acked Read 4 Done Done Sent false
18924 5808 Acked Read 4 Done Done Sent false
18925 5809 Acked Write 9 Done Done Sent false
18926 5810 Acked Write 1 Done Done Sent false
18927 5811 Acked Read 3 Done Done Sent false
18928 5812 Acked Write 10 Done Done Sent false
18929 5813 Acked Write 7 Done Done Sent false
18930 5814 Acked Read 9 Done Done Sent false
18931 5815 Acked Write 10 Done Done Sent false
18932 5816 Acked Flush 0 Done Done Sent false
18933 5817 Acked Read 2 Done Done Sent false
18934 5818 Acked Read 3 Done Done Sent false
18935 5819 Acked Write 4 Done Done Sent false
18936 5820 Acked Read 2 Done Done Sent false
18937 5821 Acked Read 1 Done Done Sent false
18938 5822 Acked Read 6 Done Done Sent false
18939 5823 Acked Write 1 Done Done Sent false
18940 5824 Acked Write 4 Done Done Sent false
18941 5825 Acked Read 3 Done Done Sent false
18942 5826 Acked Read 5 Done Done Sent false
18943 5827 Acked Read 8 Done Done Sent false
18944 5828 Acked Write 10 Done Done Sent false
18945 5829 Acked Write 6 Done Done Sent false
18946 5830 Acked Write 5 Done Done Sent false
18947 5831 Acked Read 8 Done Done Sent false
18948 5832 Acked Flush 0 Done Done Sent false
18949 5833 Acked Flush 0 Done Done Sent false
18950 5834 Acked Read 1 Done Done Sent false
18951 5835 Acked Write 5 Done Done Sent false
18952 5836 Acked Read 2 Done Done Sent false
18953 5837 Acked Flush 0 Done Done Sent false
18954 STATES DS:0 DS:1 DS:2 TOTAL
18955 Sent 0 0 302 302
18956 Done 302 302 0 604
18957 Skipped 0 0 0 0
18958 Error 0 0 0 0
18959 Last Flush: 5837 5837 5535
18960 Downstairs last five completed: 5535 5534 5533 5532 5531
18961 Upstairs last five completed: 5837 5836 5835 5834 5833
18962 CLIENT: Up:0 ds:302 act:2
18963 [8830] got start action
18964 [8830] I am going to restart
18965 Starting downstairs at port 8830
18966 Make output file at "/tmp/dsc/downstairs-8830.txt"
18967 [8830][2] reports Starting
18968 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1215)
18969 [8830][2] reports Running
189702024-11-07T14:07:18.261ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189712024-11-07T14:07:18.261ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63421) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189722024-11-07T14:07:18.262ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189732024-11-07T14:07:18.262ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189742024-11-07T14:07:18.262ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189752024-11-07T14:07:18.262ZINFOcrucible: send last flush ID to this DS: 5535 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189762024-11-07T14:07:18.262ZINFOcrucible: Replied this last flush ID: 5535 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189772024-11-07T14:07:18.262ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189782024-11-07T14:07:18.262ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189792024-11-07T14:07:18.262ZINFOcrucible: [2] client re-new 302 jobs since flush 5535 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189802024-11-07T14:07:18.263ZINFOcrucible: [2] Replayed 302 jobs since flush: 5535 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
18981 ----------------------------------------------------------------
18982 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18983 Downstairs last five completed: 5837 5836 5835 5834 5833
18984 Upstairs last five completed: 5837 5836 5835 5834 5833
18985 CLIENT: Up:0 ds:0 act:3
18986 Replay: All jobs finished
18987 Main task has work to do, go find it
18988 got dsc Stop(0)
18989 stop 0
18990 Nov 07 14:07:18.559 INFO request completed[8810] Got stop action so:false kr:false
18991 , latency_us: 356, response_code: 204, uri: /stop/cid/0, method: POST, req_id: b399803c-3cb8-4446-bc6a-93fadc2bdd76, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
189922024-11-07T14:07:18.560ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189932024-11-07T14:07:18.560ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189942024-11-07T14:07:18.560ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189952024-11-07T14:07:18.560ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189962024-11-07T14:07:18.560ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
189972024-11-07T14:07:18.560ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
18998 Nov 07 14:07:18.560 INFO request completed, latency_us: 160, response_code: 200, uri: /state/cid/0, method: GET, req_id: 79c5bbf3-3e5f-48b3-8d7b-fe2cd5b971d7, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
18999 [8810] Exited with: ExitStatus(unix_wait_status(9))
19000 [8810][0] reports Exit
19001 Nov 07 14:07:22.562 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/0, method: GET, req_id: 365e9d7b-e7b9-4218-88cd-9214a74c2aae, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
19002 001/300 Write block 685 len 24576 data: 3 3 2 2 2 2
19003 002/300 Read block 3760 len 40960
19004 003/300 Write block 3104 len 40960 data: 3 5 5 5 5 4 3 3 2 2
19005 004/300 Read block 996 len 4096
19006 005/300 Write block 2768 len 12288 data: 2 3 2
19007 006/300 Write block 2883 len 4096 data: 6
19008 007/300 Read block 1765 len 4096
19009 008/300 Read block 4069 len 24576
19010 009/300 Read block 1009 len 24576
19011 010/300 Read block 913 len 36864
19012 011/300 Write block 4143 len 40960 data: 4 4 4 5 5 7 7 8 7 6
19013 012/300 Flush
19014 013/300 Write block 4660 len 28672 data: 2 3 3 3 3 3 3
19015 014/300 Write block 4867 len 16384 data: 2 2 2 4
19016 015/300 Write block 3081 len 20480 data: 6 6 6 6 5
19017 016/300 Read block 788 len 32768
19018 017/300 Write block 1807 len 8192 data: 5 5
19019 018/300 Read block 593 len 12288
19020 019/300 Read block 1592 len 24576
19021 020/300 Read block 975 len 32768
19022 021/300 Write block 767 len 12288 data: 3 4 4
19023 022/300 Read block 1254 len 16384
19024 023/300 Read block 2940 len 24576
19025 024/300 Write block 785 len 16384 data: 2 2 2 2
19026 025/300 Flush
19027 026/300 Write block 1092 len 24576 data: 4 4 4 4 4 4
19028 027/300 Flush
19029 028/300 Flush
19030 029/300 Write block 1399 len 40960 data: 4 4 5 5 6 6 5 5 5 4
19031 030/300 Read block 752 len 20480
19032 031/300 Write block 1750 len 20480 data: 6 7 6 6 5
19033 032/300 Write block 3786 len 24576 data: 4 4 4 4 4 4
19034 033/300 Read block 4061 len 12288
19035 034/300 Read block 3903 len 32768
19036 035/300 Read block 1231 len 12288
19037 036/300 Read block 872 len 28672
19038 037/300 Read block 2533 len 12288
19039 038/300 Read block 3775 len 4096
19040 039/300 Read block 2717 len 24576
19041 040/300 Write block 1022 len 28672 data: 2 3 3 3 3 3 3
19042 041/300 Write block 4504 len 16384 data: 4 2 3 4
19043 042/300 Write block 1857 len 4096 data: 6
19044 043/300 Write block 2117 len 20480 data: 2 2 2 2 2
19045 044/300 Write block 2643 len 32768 data: 6 4 4 4 4 5 5 6
19046 045/300 Read block 2076 len 12288
19047 046/300 Write block 2942 len 32768 data: 5 5 5 5 5 5 5 4
19048 047/300 Write block 2506 len 36864 data: 2 2 3 3 3 3 3 2 2
19049 048/300 Read block 2718 len 28672
19050 049/300 Write block 1710 len 24576 data: 4 4 3 3 3 4
19051 050/300 Write block 161 len 16384 data: 6 5 5 4
19052 051/300 Read block 450 len 36864
19053 052/300 Write block 4953 len 32768 data: 6 7 7 6 7 6 6 6
19054 053/300 Read block 4160 len 20480
19055 054/300 Write block 3696 len 36864 data: 7 6 7 5 4 4 6 7 7
19056 055/300 Read block 2695 len 36864
19057 056/300 Read block 3071 len 4096
19058 057/300 Flush
19059 058/300 Write block 2269 len 28672 data: 4 5 5 5 5 5 6
19060 059/300 Write block 3279 len 36864 data: 5 5 4 5 6 4 6 5 4
19061 060/300 Write block 2683 len 32768 data: 3 3 3 3 2 2 2 3
19062 061/300 Write block 4169 len 20480 data: 4 4 6 6 7
19063 062/300 Write block 802 len 20480 data: 2 2 2 3 3
19064 063/300 Read block 2171 len 12288
19065 064/300 Read block 334 len 8192
19066 065/300 Flush
19067 066/300 Read block 571 len 4096
19068 067/300 Write block 3 len 36864 data: 5 5 4 4 3 3 2 3 3
19069 068/300 Read block 4840 len 28672
19070 069/300 Read block 1410 len 20480
19071 070/300 Read block 2022 len 4096
19072 071/300 Read block 2024 len 16384
19073 072/300 Flush
19074 073/300 Read block 4844 len 4096
19075 074/300 Read block 2790 len 24576
19076 075/300 Write block 1385 len 16384 data: 4 4 4 4
19077 076/300 Write block 3677 len 32768 data: 3 2 2 2 3 3 4 4
19078 077/300 Read block 1520 len 32768
19079 078/300 Write block 4640 len 40960 data: 4 5 5 6 5 5 5 6 6 4
19080 079/300 Read block 2165 len 32768
19081 080/300 Write block 3872 len 12288 data: 4 5 5
19082 081/300 Flush
19083 082/300 Write block 4345 len 20480 data: 4 4 5 6 6
19084 083/300 Read block 2521 len 40960
19085 084/300 Read block 1974 len 4096
19086 085/300 Read block 308 len 36864
19087 086/300 Read block 3359 len 36864
19088 087/300 Read block 4582 len 12288
19089 088/300 Write block 4837 len 8192 data: 3 3
19090 089/300 Write block 1143 len 32768 data: 5 5 4 4 2 2 2 2
19091 090/300 Write block 4211 len 20480 data: 3 4 4 3 3
19092 091/300 Flush
19093 092/300 Read block 4956 len 4096
19094 093/300 Read block 965 len 40960
19095 094/300 Read block 4895 len 16384
19096 095/300 Write block 2158 len 20480 data: 5 4 3 3 4
19097 096/300 Write block 3745 len 24576 data: 4 2 3 4 4 4
19098 097/300 Flush
19099 098/300 Read block 1024 len 40960
19100 099/300 Write block 1749 len 16384 data: 4 7 8 7
19101 100/300 Read block 804 len 24576
19102 101/300 Write block 2192 len 16384 data: 4 5 4 4
19103 102/300 Read block 4263 len 20480
19104 103/300 Read block 2717 len 16384
19105 104/300 Read block 750 len 24576
19106 105/300 Write block 2701 len 40960 data: 4 4 4 3 2 2 2 2 3 4
19107 106/300 Read block 2842 len 12288
19108 107/300 Write block 1566 len 28672 data: 6 6 5 4 4 3 4
19109 108/300 Flush
19110 109/300 Flush
19111 110/300 Write block 3863 len 16384 data: 8 8 7 6
19112 111/300 Write block 2110 len 24576 data: 3 3 4 3 3 3
19113 112/300 Write block 527 len 32768 data: 5 4 4 3 5 5 5 4
19114 113/300 Read block 1857 len 24576
19115 114/300 Write block 3124 len 32768 data: 4 4 4 4 4 3 5 5
19116 115/300 Read block 3491 len 36864
19117 116/300 Write block 1431 len 24576 data: 4 4 4 4 3 4
19118 117/300 Read block 767 len 40960
19119 118/300 Read block 673 len 24576
19120 119/300 Flush
19121 120/300 Read block 3055 len 16384
19122 121/300 Write block 1613 len 4096 data: 2
19123 122/300 Read block 2610 len 36864
19124 123/300 Read block 873 len 8192
19125 124/300 Read block 474 len 8192
19126 125/300 Read block 4577 len 16384
19127 126/300 Write block 4426 len 4096 data: 3
19128 127/300 Flush
19129 128/300 Write block 756 len 20480 data: 5 3 3 3 4
19130 129/300 Flush
19131 130/300 Write block 3110 len 8192 data: 4 4
19132 131/300 Read block 1829 len 40960
19133 132/300 Read block 1651 len 20480
19134 133/300 Read block 3316 len 4096
19135 134/300 Write block 4955 len 32768 data: 8 7 8 7 7 7 5 5
19136 135/300 Read block 615 len 32768
19137 136/300 Read block 2786 len 32768
19138 137/300 Write block 2342 len 20480 data: 3 3 3 3 3
19139 138/300 Write block 1927 len 36864 data: 6 6 4 4 4 4 3 4 3
19140 139/300 Read block 4749 len 8192
19141 140/300 Read block 3581 len 12288
19142 141/300 Write block 3545 len 20480 data: 3 3 2 3 3
19143 142/300 Write block 2441 len 40960 data: 4 4 4 4 5 6 4 5 5 6
19144 143/300 Write block 360 len 8192 data: 2 2
19145 144/300 Read block 2391 len 36864
19146 145/300 Read block 1424 len 20480
19147 146/300 Read block 507 len 40960
19148 147/300 Write block 3902 len 12288 data: 4 6 6
19149 148/300 Write block 3727 len 8192 data: 3 2
19150 149/300 Read block 2627 len 4096
19151 150/300 Write block 1259 len 12288 data: 4 4 3
19152 151/300 Write block 301 len 16384 data: 6 4 4 5
19153 152/300 Read block 1208 len 28672
19154 153/300 Write block 2228 len 16384 data: 4 4 4 4
19155 154/300 Read block 4297 len 40960
19156 155/300 Read block 4911 len 8192
19157 156/300 Write block 2181 len 40960 data: 5 5 5 6 7 6 5 4 4 4
19158 157/300 Read block 980 len 24576
19159 158/300 Write block 3105 len 4096 data: 6
19160 159/300 Read block 3596 len 28672
19161 160/300 Read block 4314 len 20480
19162 161/300 Write block 459 len 36864 data: 3 3 3 3 3 3 3 5 5
19163 162/300 Write block 1117 len 28672 data: 4 4 4 4 4 4 4
19164 163/300 Read block 4403 len 4096
19165 164/300 Write block 1862 len 16384 data: 4 4 3 3
19166 165/300 Write block 2639 len 4096 data: 3
19167 166/300 Read block 1215 len 20480
19168 167/300 Write block 625 len 12288 data: 3 4 3
19169 168/300 Read block 415 len 8192
19170 169/300 Read block 1103 len 20480
19171 170/300 Read block 4507 len 12288
19172 171/300 Read block 3158 len 40960
19173 172/300 Read block 2544 len 24576
19174 173/300 Read block 2627 len 36864
19175 174/300 Write block 4698 len 12288 data: 5 4 4
19176 175/300 Read block 87 len 16384
19177 176/300 Read block 3394 len 8192
19178 177/300 Read block 3312 len 4096
19179 178/300 Write block 574 len 24576 data: 3 4 4 4 4 3
19180 179/300 Read block 2963 len 4096
19181 180/300 Read block 1707 len 32768
19182 181/300 Flush
19183 182/300 Write block 1258 len 4096 data: 4
19184 183/300 Write block 4813 len 32768 data: 4 4 3 3 3 4 3 3
19185 184/300 Write block 4997 len 8192 data: 2 2
19186 185/300 Read block 4961 len 24576
19187 186/300 Write block 4743 len 40960 data: 4 4 4 5 3 3 3 4 5 4
19188 187/300 Write block 3028 len 28672 data: 6 6 5 5 5 4 4
19189 188/300 Read block 775 len 36864
19190 189/300 Write block 4529 len 12288 data: 5 6 6
19191 190/300 Read block 1669 len 8192
19192 191/300 Write block 1475 len 28672 data: 4 3 3 4 4 5 5
19193 192/300 Write block 226 len 8192 data: 7 8
19194 193/300 Read block 869 len 40960
19195 194/300 Read block 2717 len 8192
19196 195/300 Read block 4847 len 36864
19197 196/300 Read block 493 len 12288
19198 197/300 Read block 2730 len 32768
19199 198/300 Read block 4637 len 24576
19200 199/300 Read block 2236 len 20480
19201 200/300 Read block 2581 len 8192
19202 201/300 Read block 1863 len 12288
19203 202/300 Read block 3846 len 40960
19204 203/300 Read block 4162 len 40960
19205 204/300 Write block 1636 len 40960 data: 4 4 4 4 4 3 3 3 4 4
19206 205/300 Read block 1364 len 40960
19207 206/300 Read block 3801 len 8192
19208 207/300 Write block 3426 len 16384 data: 6 6 5 5
19209 208/300 Read block 4411 len 20480
19210 209/300 Write block 1330 len 20480 data: 6 5 6 7 5
19211 210/300 Read block 3131 len 16384
19212 211/300 Write block 4710 len 32768 data: 3 3 4 4 3 3 3 3
19213 212/300 Read block 4258 len 32768
19214 213/300 Read block 4683 len 24576
19215 214/300 Read block 808 len 4096
19216 215/300 Write block 1213 len 36864 data: 5 5 4 5 5 5 5 4 4
19217 216/300 Write block 4731 len 16384 data: 7 6 7 5
19218 217/300 Read block 4705 len 8192
19219 218/300 Read block 3518 len 8192
19220 219/300 Write block 1852 len 4096 data: 7
19221 220/300 Read block 2934 len 12288
19222 221/300 Read block 4330 len 28672
19223 222/300 Read block 2081 len 20480
19224 223/300 Write block 1669 len 16384 data: 6 6 4 4
19225 224/300 Write block 4491 len 12288 data: 5 4 3
19226 225/300 Read block 1617 len 4096
19227 226/300 Write block 3198 len 32768 data: 3 3 4 4 4 4 4 4
19228 227/300 Flush
19229 228/300 Read block 531 len 36864
19230 229/300 Read block 308 len 4096
19231 230/300 Read block 1390 len 16384
19232 231/300 Read block 3128 len 32768
19233 232/300 Write block 4842 len 4096 data: 7
19234 233/300 Write block 3472 len 8192 data: 4 4
19235 234/300 Read block 536 len 36864
19236 235/300 Write block 2638 len 20480 data: 3 4 4 5 6
19237 236/300 Read block 2870 len 28672
19238 237/300 Read block 1909 len 16384
19239 238/300 Write block 342 len 4096 data: 4
19240 239/300 Read block 399 len 12288
19241 240/300 Write block 385 len 12288 data: 4 4 4
19242 241/300 Read block 358 len 20480
19243 242/300 Read block 2372 len 16384
19244 243/300 Write block 465 len 40960 data: 4 6 6 5 5 5 3 3 3 3
19245 244/300 Read block 542 len 36864
19246 245/300 Write block 1573 len 20480 data: 5 5 5 5 3
19247 246/300 Write block 3256 len 4096 data: 2
19248 247/300 Write block 4237 len 40960 data: 4 5 6 5 5 4 3 2 2 2
19249 248/300 Flush
19250 249/300 Read block 2497 len 12288
19251 250/300 Flush
19252 251/300 Write block 1324 len 36864 data: 3 5 4 5 5 6 7 6 7
19253 252/300 Read block 1923 len 4096
19254 253/300 Write block 4906 len 40960 data: 4 4 5 5 4 5 5 5 5 6
19255 254/300 Read block 2753 len 4096
19256 255/300 Write block 3512 len 20480 data: 6 5 5 3 3
19257 256/300 Flush
19258 257/300 Read block 1753 len 28672
19259 258/300 Read block 3741 len 24576
19260 259/300 Write block 1333 len 20480 data: 8 6 6 6 6
19261 260/300 Read block 4579 len 4096
19262 261/300 Write block 2783 len 36864 data: 5 5 5 6 5 5 6 6 6
19263 262/300 Write block 4452 len 8192 data: 4 4
19264 263/300 Read block 4081 len 32768
19265 264/300 Write block 4841 len 28672 data: 5 8 7 7 7 7 7
19266 265/300 Read block 4620 len 12288
19267 266/300 Write block 3921 len 12288 data: 4 4 6
19268 267/300 Read block 1580 len 40960
19269 268/300 Read block 2030 len 20480
19270 269/300 Read block 138 len 8192
19271 270/300 Write block 1046 len 12288 data: 6 6 5
19272 271/300 Read block 4577 len 40960
19273 272/300 Read block 3484 len 40960
19274 273/300 Read block 71 len 16384
19275 274/300 Read block 3286 len 40960
19276 275/300 Read block 1119 len 4096
19277 276/300 Read block 1279 len 4096
19278 277/300 Write block 342 len 24576 data: 5 4 5 6 4 4
19279 278/300 Read block 4403 len 4096
19280 279/300 Write block 3409 len 24576 data: 3 3 4 4 4 4
19281 280/300 Write block 2662 len 40960 data: 5 5 5 5 4 3 4 4 5 5
19282 281/300 Write block 3014 len 16384 data: 4 4 4 6
19283 282/300 Write block 3708 len 24576 data: 5 4 5 6 5 4
19284 283/300 Write block 3738 len 16384 data: 5 6 4 4
19285 284/300 Write block 2635 len 4096 data: 4
19286 285/300 Read block 3921 len 32768
19287 286/300 Write block 3783 len 20480 data: 2 4 4 5 5
19288 287/300 Write block 3541 len 36864 data: 3 4 4 3 4 4 3 4 4
19289 288/300 Read block 3979 len 20480
19290 289/300 Write block 1626 len 28672 data: 3 3 3 2 2 4 3
19291 290/300 Write block 1832 len 4096 data: 5
19292 291/300 Read block 2330 len 16384
19293 292/300 Read block 4684 len 40960
19294 293/300 Read block 3572 len 28672
19295 294/300 Read block 656 len 24576
19296 295/300 Flush
19297 296/300 Write block 1393 len 24576 data: 3 2 2 3 3 3
19298 297/300 Write block 289 len 4096 data: 5
19299 298/300 Read block 85 len 20480
19300 299/300 Read block 2788 len 28672
19301 300/300 Write block 4764 len 24576 data: 5 6 5 4 3 3
19302 301/300 Read block 18 len 20480
19303 Main task has work to do, go find it
19304 got dsc Start(0)
19305 start 0
19306 Nov 07 14:07:22.837 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/0, method: POST, req_id: 6bab6a26-b36f-4dde-91e3-702e03f5ff37, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
19307 Replay: started 0, returned:Ok(())
19308 ----------------------------------------------------------------
19309 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19310 States: Offline Active Active
19311 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19312 5838 Acked Write 6 Sent Done Done false
19313 5839 Acked Read 10 Sent Done Done false
19314 5840 Acked Write 10 Sent Done Done false
19315 5841 Acked Read 1 Sent Done Done false
19316 5842 Acked Write 3 Sent Done Done false
19317 5843 Acked Write 1 Sent Done Done false
19318 5844 Acked Read 1 Sent Done Done false
19319 5845 Acked Read 6 Sent Done Done false
19320 5846 Acked Read 6 Sent Done Done false
19321 5847 Acked Read 9 Sent Done Done false
19322 5848 Acked Write 10 Sent Done Done false
19323 5849 Acked Flush 0 Sent Done Done false
19324 5850 Acked Write 7 Sent Done Done false
19325 5851 Acked Write 4 Sent Done Done false
19326 5852 Acked Write 5 Sent Done Done false
19327 5853 Acked Read 8 Sent Done Done false
19328 5854 Acked Write 2 Sent Done Done false
19329 5855 Acked Read 3 Sent Done Done false
19330 5856 Acked Read 6 Sent Done Done false
19331 5857 Acked Read 8 Sent Done Done false
19332 5858 Acked Write 3 Sent Done Done false
19333 5859 Acked Read 4 Sent Done Done false
19334 5860 Acked Read 6 Sent Done Done false
19335 5861 Acked Write 4 Sent Done Done false
19336 5862 Acked Flush 0 Sent Done Done false
19337 5863 Acked Write 6 Sent Done Done false
19338 5864 Acked Flush 0 Sent Done Done false
19339 5865 Acked Flush 0 Sent Done Done false
19340 5866 Acked Write 10 Sent Done Done false
19341 5867 Acked Read 5 Sent Done Done false
19342 5868 Acked Write 5 Sent Done Done false
19343 5869 Acked Write 6 Sent Done Done false
19344 5870 Acked Read 3 Sent Done Done false
19345 5871 Acked Read 8 Sent Done Done false
19346 5872 Acked Read 3 Sent Done Done false
19347 5873 Acked Read 7 Sent Done Done false
19348 5874 Acked Read 3 Sent Done Done false
19349 5875 Acked Read 1 Sent Done Done false
19350 5876 Acked Read 6 Sent Done Done false
19351 5877 Acked Write 7 Sent Done Done false
19352 5878 Acked Write 4 Sent Done Done false
19353 5879 Acked Write 1 Sent Done Done false
19354 5880 Acked Write 5 Sent Done Done false
19355 5881 Acked Write 8 Sent Done Done false
19356 5882 Acked Read 3 Sent Done Done false
19357 5883 Acked Write 8 Sent Done Done false
19358 5884 Acked Write 9 Sent Done Done false
19359 5885 Acked Read 7 Sent Done Done false
19360 5886 Acked Write 6 Sent Done Done false
19361 5887 Acked Write 4 Sent Done Done false
19362 5888 Acked Read 9 Sent Done Done false
19363 5889 Acked Write 8 Sent Done Done false
19364 5890 Acked Read 5 Sent Done Done false
19365 5891 Acked Write 9 Sent Done Done false
19366 5892 Acked Read 9 Sent Done Done false
19367 5893 Acked Read 1 Sent Done Done false
19368 5894 Acked Flush 0 Sent Done Done false
19369 5895 Acked Write 7 Sent Done Done false
19370 5896 Acked Write 9 Sent Done Done false
19371 5897 Acked Write 8 Sent Done Done false
19372 5898 Acked Write 5 Sent Done Done false
19373 5899 Acked Write 5 Sent Done Done false
19374 5900 Acked Read 3 Sent Done Done false
19375 5901 Acked Read 2 Sent Done Done false
19376 5902 Acked Flush 0 Sent Done Done false
19377 5903 Acked Read 1 Sent Done Done false
19378 5904 Acked Write 9 Sent Done Done false
19379 5905 Acked Read 7 Sent Done Done false
19380 5906 Acked Read 5 Sent Done Done false
19381 5907 Acked Read 1 Sent Done Done false
19382 5908 Acked Read 4 Sent Done Done false
19383 5909 Acked Flush 0 Sent Done Done false
19384 5910 Acked Read 1 Sent Done Done false
19385 5911 Acked Read 6 Sent Done Done false
19386 5912 Acked Write 4 Sent Done Done false
19387 5913 Acked Write 8 Sent Done Done false
19388 5914 Acked Read 8 Sent Done Done false
19389 5915 Acked Write 10 Sent Done Done false
19390 5916 Acked Read 8 Sent Done Done false
19391 5917 Acked Write 3 Sent Done Done false
19392 5918 Acked Flush 0 Sent Done Done false
19393 5919 Acked Write 5 Sent Done Done false
19394 5920 Acked Read 10 Sent Done Done false
19395 5921 Acked Read 1 Sent Done Done false
19396 5922 Acked Read 9 Sent Done Done false
19397 5923 Acked Read 9 Sent Done Done false
19398 5924 Acked Read 3 Sent Done Done false
19399 5925 Acked Write 2 Sent Done Done false
19400 5926 Acked Write 8 Sent Done Done false
19401 5927 Acked Write 5 Sent Done Done false
19402 5928 Acked Flush 0 Sent Done Done false
19403 5929 Acked Read 1 Sent Done Done false
19404 5930 Acked Read 10 Sent Done Done false
19405 5931 Acked Read 4 Sent Done Done false
19406 5932 Acked Write 5 Sent Done Done false
19407 5933 Acked Write 6 Sent Done Done false
19408 5934 Acked Flush 0 Sent Done Done false
19409 5935 Acked Read 10 Sent Done Done false
19410 5936 Acked Write 4 Sent Done Done false
19411 5937 Acked Read 6 Sent Done Done false
19412 5938 Acked Write 4 Sent Done Done false
19413 5939 Acked Read 5 Sent Done Done false
19414 5940 Acked Read 4 Sent Done Done false
19415 5941 Acked Read 6 Sent Done Done false
19416 5942 Acked Write 10 Sent Done Done false
19417 5943 Acked Read 3 Sent Done Done false
19418 5944 Acked Write 7 Sent Done Done false
19419 5945 Acked Flush 0 Sent Done Done false
19420 5946 Acked Flush 0 Sent Done Done false
19421 5947 Acked Write 4 Sent Done Done false
19422 5948 Acked Write 6 Sent Done Done false
19423 5949 Acked Write 8 Sent Done Done false
19424 5950 Acked Read 6 Sent Done Done false
19425 5951 Acked Write 8 Sent Done Done false
19426 5952 Acked Read 9 Sent Done Done false
19427 5953 Acked Write 6 Sent Done Done false
19428 5954 Acked Read 10 Sent Done Done false
19429 5955 Acked Read 6 Sent Done Done false
19430 5956 Acked Flush 0 Sent Done Done false
19431 5957 Acked Read 4 Sent Done Done false
19432 5958 Acked Write 1 Sent Done Done false
19433 5959 Acked Read 9 Sent Done Done false
19434 5960 Acked Read 2 Sent Done Done false
19435 5961 Acked Read 2 Sent Done Done false
19436 5962 Acked Read 4 Sent Done Done false
19437 5963 Acked Write 1 Sent Done Done false
19438 5964 Acked Flush 0 Sent Done Done false
19439 5965 Acked Write 5 Sent Done Done false
19440 5966 Acked Flush 0 Sent Done Done false
19441 5967 Acked Write 2 Sent Done Done false
19442 5968 Acked Read 10 Sent Done Done false
19443 5969 Acked Read 5 Sent Done Done false
19444 5970 Acked Read 1 Sent Done Done false
19445 5971 Acked Write 8 Sent Done Done false
19446 5972 Acked Read 8 Sent Done Done false
19447 5973 Acked Read 8 Sent Done Done false
19448 5974 Acked Write 5 Sent Done Done false
19449 5975 Acked Write 9 Sent Done Done false
19450 5976 Acked Read 2 Sent Done Done false
19451 5977 Acked Read 3 Sent Done Done false
19452 5978 Acked Write 5 Sent Done Done false
19453 5979 Acked Write 10 Sent Done Done false
19454 5980 Acked Write 2 Sent Done Done false
19455 5981 Acked Read 9 Sent Done Done false
19456 5982 Acked Read 5 Sent Done Done false
19457 5983 Acked Read 10 Sent Done Done false
19458 5984 Acked Write 3 Sent Done Done false
19459 5985 Acked Write 2 Sent Done Done false
19460 5986 Acked Read 1 Sent Done Done false
19461 5987 Acked Write 3 Sent Done Done false
19462 5988 Acked Write 4 Sent Done Done false
19463 5989 Acked Read 7 Sent Done Done false
19464 5990 Acked Write 4 Sent Done Done false
19465 5991 Acked Read 10 Sent Done Done false
19466 5992 Acked Read 2 Sent Done Done false
19467 5993 Acked Write 10 Sent Done Done false
19468 5994 Acked Read 6 Sent Done Done false
19469 5995 Acked Write 1 Sent Done Done false
19470 5996 Acked Read 7 Sent Done Done false
19471 5997 Acked Read 5 Sent Done Done false
19472 5998 Acked Write 9 Sent Done Done false
19473 5999 Acked Write 7 Sent Done Done false
19474 6000 Acked Read 1 Sent Done Done false
19475 6001 Acked Write 4 Sent Done Done false
19476 6002 Acked Write 1 Sent Done Done false
19477 6003 Acked Read 5 Sent Done Done false
19478 6004 Acked Write 3 Sent Done Done false
19479 6005 Acked Read 2 Sent Done Done false
19480 6006 Acked Read 5 Sent Done Done false
19481 6007 Acked Read 3 Sent Done Done false
19482 6008 Acked Read 10 Sent Done Done false
19483 6009 Acked Read 6 Sent Done Done false
19484 6010 Acked Read 9 Sent Done Done false
19485 6011 Acked Write 3 Sent Done Done false
19486 6012 Acked Read 4 Sent Done Done false
19487 6013 Acked Read 2 Sent Done Done false
19488 6014 Acked Read 1 Sent Done Done false
19489 6015 Acked Write 6 Sent Done Done false
19490 6016 Acked Read 1 Sent Done Done false
19491 6017 Acked Read 8 Sent Done Done false
19492 6018 Acked Flush 0 Sent Done Done false
19493 6019 Acked Write 1 Sent Done Done false
19494 6020 Acked Write 8 Sent Done Done false
19495 6021 Acked Write 2 Sent Done Done false
19496 6022 Acked Read 6 Sent Done Done false
19497 6023 Acked Write 10 Sent Done Done false
19498 6024 Acked Write 7 Sent Done Done false
19499 6025 Acked Read 9 Sent Done Done false
19500 6026 Acked Write 3 Sent Done Done false
19501 6027 Acked Read 2 Sent Done Done false
19502 6028 Acked Write 7 Sent Done Done false
19503 6029 Acked Write 2 Sent Done Done false
19504 6030 Acked Read 10 Sent Done Done false
19505 6031 Acked Read 2 Sent Done Done false
19506 6032 Acked Read 9 Sent Done Done false
19507 6033 Acked Read 3 Sent Done Done false
19508 6034 Acked Read 8 Sent Done Done false
19509 6035 Acked Read 6 Sent Done Done false
19510 6036 Acked Read 5 Sent Done Done false
19511 6037 Acked Read 2 Sent Done Done false
19512 6038 Acked Read 3 Sent Done Done false
19513 6039 Acked Read 10 Sent Done Done false
19514 6040 Acked Read 10 Sent Done Done false
19515 6041 Acked Write 10 Sent Done Done false
19516 6042 Acked Read 10 Sent Done Done false
19517 6043 Acked Read 2 Sent Done Done false
19518 6044 Acked Write 4 Sent Done Done false
19519 6045 Acked Read 5 Sent Done Done false
19520 6046 Acked Write 5 Sent Done Done false
19521 6047 Acked Read 4 Sent Done Done false
19522 6048 Acked Write 8 Sent Done Done false
19523 6049 Acked Read 8 Sent Done Done false
19524 6050 Acked Read 6 Sent Done Done false
19525 6051 Acked Read 1 Sent Done Done false
19526 6052 Acked Write 9 Sent Done Done false
19527 6053 Acked Write 4 Sent Done Done false
19528 6054 Acked Read 2 Sent Done Done false
19529 6055 Acked Read 2 Sent Done Done false
19530 6056 Acked Write 1 Sent Done Done false
19531 6057 Acked Read 3 Sent Done Done false
19532 6058 Acked Read 7 Sent Done Done false
19533 6059 Acked Read 5 Sent Done Done false
19534 6060 Acked Write 4 Sent Done Done false
19535 6061 Acked Write 3 Sent Done Done false
19536 6062 Acked Read 1 Sent Done Done false
19537 6063 Acked Write 8 Sent Done Done false
19538 6064 Acked Flush 0 Sent Done Done false
19539 6065 Acked Read 9 Sent Done Done false
19540 6066 Acked Read 1 Sent Done Done false
19541 6067 Acked Read 4 Sent Done Done false
19542 6068 Acked Read 8 Sent Done Done false
19543 6069 Acked Write 1 Sent Done Done false
19544 6070 Acked Write 2 Sent Done Done false
19545 6071 Acked Read 9 Sent Done Done false
19546 6072 Acked Write 5 Sent Done Done false
19547 6073 Acked Read 7 Sent Done Done false
19548 6074 Acked Read 4 Sent Done Done false
19549 6075 Acked Write 1 Sent Done Done false
19550 6076 Acked Read 3 Sent Done Done false
19551 6077 Acked Write 3 Sent Done Done false
19552 6078 Acked Read 5 Sent Done Done false
19553 6079 Acked Read 4 Sent Done Done false
19554 6080 Acked Write 10 Sent Done Done false
19555 6081 Acked Read 9 Sent Done Done false
19556 6082 Acked Write 5 Sent Done Done false
19557 6083 Acked Write 1 Sent Done Done false
19558 6084 Acked Write 10 Sent Done Done false
19559 6085 Acked Flush 0 Sent Done Done false
19560 6086 Acked Read 3 Sent Done Done false
19561 6087 Acked Flush 0 Sent Done Done false
19562 6088 Acked Write 9 Sent Done Done false
19563 6089 Acked Read 1 Sent Done Done false
19564 6090 Acked Write 10 Sent Done Done false
19565 6091 Acked Read 1 Sent Done Done false
19566 6092 Acked Write 5 Sent Done Done false
19567 6093 Acked Flush 0 Sent Done Done false
19568 6094 Acked Read 7 Sent Done Done false
19569 6095 Acked Read 6 Sent Done Done false
19570 6096 Acked Write 5 Sent Done Done false
19571 6097 Acked Read 1 Sent Done Done false
19572 6098 Acked Write 9 Sent Done Done false
19573 6099 Acked Write 2 Sent Done Done false
19574 6100 Acked Read 8 Sent Done Done false
19575 6101 Acked Write 7 Sent Done Done false
19576 6102 Acked Read 3 Sent Done Done false
19577 6103 Acked Write 3 Sent Done Done false
19578 6104 Acked Read 10 Sent Done Done false
19579 6105 Acked Read 5 Sent Done Done false
19580 6106 Acked Read 2 Sent Done Done false
19581 6107 Acked Write 3 Sent Done Done false
19582 6108 Acked Read 10 Sent Done Done false
19583 6109 Acked Read 10 Sent Done Done false
19584 6110 Acked Read 4 Sent Done Done false
19585 6111 Acked Read 10 Sent Done Done false
19586 6112 Acked Read 1 Sent Done Done false
19587 6113 Acked Read 1 Sent Done Done false
19588 6114 Acked Write 6 Sent Done Done false
19589 6115 Acked Read 1 Sent Done Done false
19590 6116 Acked Write 6 Sent Done Done false
19591 6117 Acked Write 10 Sent Done Done false
19592 6118 Acked Write 4 Sent Done Done false
19593 6119 Acked Write 6 Sent Done Done false
19594 6120 Acked Write 4 Sent Done Done false
19595 6121 Acked Write 1 Sent Done Done false
19596 6122 Acked Read 8 Sent Done Done false
19597 6123 Acked Write 5 Sent Done Done false
19598 6124 Acked Write 9 Sent Done Done false
19599 6125 Acked Read 5 Sent Done Done false
19600 6126 Acked Write 7 Sent Done Done false
19601 6127 Acked Write 1 Sent Done Done false
19602 6128 Acked Read 4 Sent Done Done false
19603 6129 Acked Read 10 Sent Done Done false
19604 6130 Acked Read 7 Sent Done Done false
19605 6131 Acked Read 6 Sent Done Done false
19606 6132 Acked Flush 0 Sent Done Done false
19607 6133 Acked Write 6 Sent Done Done false
19608 6134 Acked Write 1 Sent Done Done false
19609 6135 Acked Read 5 Sent Done Done false
19610 6136 Acked Read 7 Sent Done Done false
19611 6137 Acked Write 6 Sent Done Done false
19612 6138 Acked Read 5 Sent Done Done false
19613 STATES DS:0 DS:1 DS:2 TOTAL
19614 Sent 301 0 0 301
19615 Done 0 301 301 602
19616 Skipped 0 0 0 0
19617 Error 0 0 0 0
19618 Last Flush: 5837 6132 6132
19619 Downstairs last five completed: 5837 5836 5835 5834 5833
19620 Upstairs last five completed: 6138 6137 6136 6135 6134
19621 CLIENT: Up:0 ds:301 act:2
19622 [8810] got start action
19623 [8810] I am going to restart
19624 Starting downstairs at port 8810
19625 Make output file at "/tmp/dsc/downstairs-8810.txt"
19626 [8810][0] reports Starting
19627 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1216)
19628 [8810][0] reports Running
19629 ----------------------------------------------------------------
19630 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19631 States: Offline Active Active
19632 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19633 5838 Acked Write 6 Sent Done Done false
19634 5839 Acked Read 10 Sent Done Done false
19635 5840 Acked Write 10 Sent Done Done false
19636 5841 Acked Read 1 Sent Done Done false
19637 5842 Acked Write 3 Sent Done Done false
19638 5843 Acked Write 1 Sent Done Done false
19639 5844 Acked Read 1 Sent Done Done false
19640 5845 Acked Read 6 Sent Done Done false
19641 5846 Acked Read 6 Sent Done Done false
19642 5847 Acked Read 9 Sent Done Done false
19643 5848 Acked Write 10 Sent Done Done false
19644 5849 Acked Flush 0 Sent Done Done false
19645 5850 Acked Write 7 Sent Done Done false
19646 5851 Acked Write 4 Sent Done Done false
19647 5852 Acked Write 5 Sent Done Done false
19648 5853 Acked Read 8 Sent Done Done false
19649 5854 Acked Write 2 Sent Done Done false
19650 5855 Acked Read 3 Sent Done Done false
19651 5856 Acked Read 6 Sent Done Done false
19652 5857 Acked Read 8 Sent Done Done false
19653 5858 Acked Write 3 Sent Done Done false
19654 5859 Acked Read 4 Sent Done Done false
19655 5860 Acked Read 6 Sent Done Done false
19656 5861 Acked Write 4 Sent Done Done false
19657 5862 Acked Flush 0 Sent Done Done false
19658 5863 Acked Write 6 Sent Done Done false
19659 5864 Acked Flush 0 Sent Done Done false
19660 5865 Acked Flush 0 Sent Done Done false
19661 5866 Acked Write 10 Sent Done Done false
19662 5867 Acked Read 5 Sent Done Done false
19663 5868 Acked Write 5 Sent Done Done false
19664 5869 Acked Write 6 Sent Done Done false
19665 5870 Acked Read 3 Sent Done Done false
19666 5871 Acked Read 8 Sent Done Done false
19667 5872 Acked Read 3 Sent Done Done false
19668 5873 Acked Read 7 Sent Done Done false
19669 5874 Acked Read 3 Sent Done Done false
19670 5875 Acked Read 1 Sent Done Done false
19671 5876 Acked Read 6 Sent Done Done false
19672 5877 Acked Write 7 Sent Done Done false
19673 5878 Acked Write 4 Sent Done Done false
19674 5879 Acked Write 1 Sent Done Done false
19675 5880 Acked Write 5 Sent Done Done false
19676 5881 Acked Write 8 Sent Done Done false
19677 5882 Acked Read 3 Sent Done Done false
19678 5883 Acked Write 8 Sent Done Done false
19679 5884 Acked Write 9 Sent Done Done false
19680 5885 Acked Read 7 Sent Done Done false
19681 5886 Acked Write 6 Sent Done Done false
19682 5887 Acked Write 4 Sent Done Done false
19683 5888 Acked Read 9 Sent Done Done false
19684 5889 Acked Write 8 Sent Done Done false
19685 5890 Acked Read 5 Sent Done Done false
19686 5891 Acked Write 9 Sent Done Done false
19687 5892 Acked Read 9 Sent Done Done false
19688 5893 Acked Read 1 Sent Done Done false
19689 5894 Acked Flush 0 Sent Done Done false
19690 5895 Acked Write 7 Sent Done Done false
19691 5896 Acked Write 9 Sent Done Done false
19692 5897 Acked Write 8 Sent Done Done false
19693 5898 Acked Write 5 Sent Done Done false
19694 5899 Acked Write 5 Sent Done Done false
19695 5900 Acked Read 3 Sent Done Done false
19696 5901 Acked Read 2 Sent Done Done false
19697 5902 Acked Flush 0 Sent Done Done false
19698 5903 Acked Read 1 Sent Done Done false
19699 5904 Acked Write 9 Sent Done Done false
19700 5905 Acked Read 7 Sent Done Done false
19701 5906 Acked Read 5 Sent Done Done false
19702 5907 Acked Read 1 Sent Done Done false
19703 5908 Acked Read 4 Sent Done Done false
19704 5909 Acked Flush 0 Sent Done Done false
19705 5910 Acked Read 1 Sent Done Done false
19706 5911 Acked Read 6 Sent Done Done false
19707 5912 Acked Write 4 Sent Done Done false
19708 5913 Acked Write 8 Sent Done Done false
19709 5914 Acked Read 8 Sent Done Done false
19710 5915 Acked Write 10 Sent Done Done false
19711 5916 Acked Read 8 Sent Done Done false
19712 5917 Acked Write 3 Sent Done Done false
19713 5918 Acked Flush 0 Sent Done Done false
19714 5919 Acked Write 5 Sent Done Done false
19715 5920 Acked Read 10 Sent Done Done false
19716 5921 Acked Read 1 Sent Done Done false
19717 5922 Acked Read 9 Sent Done Done false
19718 5923 Acked Read 9 Sent Done Done false
19719 5924 Acked Read 3 Sent Done Done false
19720 5925 Acked Write 2 Sent Done Done false
19721 5926 Acked Write 8 Sent Done Done false
19722 5927 Acked Write 5 Sent Done Done false
19723 5928 Acked Flush 0 Sent Done Done false
19724 5929 Acked Read 1 Sent Done Done false
19725 5930 Acked Read 10 Sent Done Done false
19726 5931 Acked Read 4 Sent Done Done false
19727 5932 Acked Write 5 Sent Done Done false
19728 5933 Acked Write 6 Sent Done Done false
19729 5934 Acked Flush 0 Sent Done Done false
19730 5935 Acked Read 10 Sent Done Done false
19731 5936 Acked Write 4 Sent Done Done false
19732 5937 Acked Read 6 Sent Done Done false
19733 5938 Acked Write 4 Sent Done Done false
19734 5939 Acked Read 5 Sent Done Done false
19735 5940 Acked Read 4 Sent Done Done false
19736 5941 Acked Read 6 Sent Done Done false
19737 5942 Acked Write 10 Sent Done Done false
19738 5943 Acked Read 3 Sent Done Done false
19739 5944 Acked Write 7 Sent Done Done false
19740 5945 Acked Flush 0 Sent Done Done false
19741 5946 Acked Flush 0 Sent Done Done false
19742 5947 Acked Write 4 Sent Done Done false
19743 5948 Acked Write 6 Sent Done Done false
19744 5949 Acked Write 8 Sent Done Done false
19745 5950 Acked Read 6 Sent Done Done false
19746 5951 Acked Write 8 Sent Done Done false
19747 5952 Acked Read 9 Sent Done Done false
19748 5953 Acked Write 6 Sent Done Done false
19749 5954 Acked Read 10 Sent Done Done false
19750 5955 Acked Read 6 Sent Done Done false
19751 5956 Acked Flush 0 Sent Done Done false
19752 5957 Acked Read 4 Sent Done Done false
19753 5958 Acked Write 1 Sent Done Done false
19754 5959 Acked Read 9 Sent Done Done false
19755 5960 Acked Read 2 Sent Done Done false
19756 5961 Acked Read 2 Sent Done Done false
19757 5962 Acked Read 4 Sent Done Done false
19758 5963 Acked Write 1 Sent Done Done false
19759 5964 Acked Flush 0 Sent Done Done false
19760 5965 Acked Write 5 Sent Done Done false
19761 5966 Acked Flush 0 Sent Done Done false
19762 5967 Acked Write 2 Sent Done Done false
19763 5968 Acked Read 10 Sent Done Done false
19764 5969 Acked Read 5 Sent Done Done false
19765 5970 Acked Read 1 Sent Done Done false
19766 5971 Acked Write 8 Sent Done Done false
19767 5972 Acked Read 8 Sent Done Done false
19768 5973 Acked Read 8 Sent Done Done false
19769 5974 Acked Write 5 Sent Done Done false
19770 5975 Acked Write 9 Sent Done Done false
19771 5976 Acked Read 2 Sent Done Done false
19772 5977 Acked Read 3 Sent Done Done false
19773 5978 Acked Write 5 Sent Done Done false
19774 5979 Acked Write 10 Sent Done Done false
19775 5980 Acked Write 2 Sent Done Done false
19776 5981 Acked Read 9 Sent Done Done false
19777 5982 Acked Read 5 Sent Done Done false
19778 5983 Acked Read 10 Sent Done Done false
19779 5984 Acked Write 3 Sent Done Done false
19780 5985 Acked Write 2 Sent Done Done false
19781 5986 Acked Read 1 Sent Done Done false
19782 5987 Acked Write 3 Sent Done Done false
19783 5988 Acked Write 4 Sent Done Done false
19784 5989 Acked Read 7 Sent Done Done false
19785 5990 Acked Write 4 Sent Done Done false
19786 5991 Acked Read 10 Sent Done Done false
19787 5992 Acked Read 2 Sent Done Done false
19788 5993 Acked Write 10 Sent Done Done false
19789 5994 Acked Read 6 Sent Done Done false
19790 5995 Acked Write 1 Sent Done Done false
19791 5996 Acked Read 7 Sent Done Done false
19792 5997 Acked Read 5 Sent Done Done false
19793 5998 Acked Write 9 Sent Done Done false
19794 5999 Acked Write 7 Sent Done Done false
19795 6000 Acked Read 1 Sent Done Done false
19796 6001 Acked Write 4 Sent Done Done false
19797 6002 Acked Write 1 Sent Done Done false
19798 6003 Acked Read 5 Sent Done Done false
19799 6004 Acked Write 3 Sent Done Done false
19800 6005 Acked Read 2 Sent Done Done false
19801 6006 Acked Read 5 Sent Done Done false
19802 6007 Acked Read 3 Sent Done Done false
19803 6008 Acked Read 10 Sent Done Done false
19804 6009 Acked Read 6 Sent Done Done false
19805 6010 Acked Read 9 Sent Done Done false
19806 6011 Acked Write 3 Sent Done Done false
19807 6012 Acked Read 4 Sent Done Done false
19808 6013 Acked Read 2 Sent Done Done false
19809 6014 Acked Read 1 Sent Done Done false
19810 6015 Acked Write 6 Sent Done Done false
19811 6016 Acked Read 1 Sent Done Done false
19812 6017 Acked Read 8 Sent Done Done false
19813 6018 Acked Flush 0 Sent Done Done false
19814 6019 Acked Write 1 Sent Done Done false
19815 6020 Acked Write 8 Sent Done Done false
19816 6021 Acked Write 2 Sent Done Done false
19817 6022 Acked Read 6 Sent Done Done false
19818 6023 Acked Write 10 Sent Done Done false
19819 6024 Acked Write 7 Sent Done Done false
19820 6025 Acked Read 9 Sent Done Done false
19821 6026 Acked Write 3 Sent Done Done false
19822 6027 Acked Read 2 Sent Done Done false
19823 6028 Acked Write 7 Sent Done Done false
19824 6029 Acked Write 2 Sent Done Done false
19825 6030 Acked Read 10 Sent Done Done false
19826 6031 Acked Read 2 Sent Done Done false
19827 6032 Acked Read 9 Sent Done Done false
19828 6033 Acked Read 3 Sent Done Done false
19829 6034 Acked Read 8 Sent Done Done false
19830 6035 Acked Read 6 Sent Done Done false
19831 6036 Acked Read 5 Sent Done Done false
19832 6037 Acked Read 2 Sent Done Done false
19833 6038 Acked Read 3 Sent Done Done false
19834 6039 Acked Read 10 Sent Done Done false
19835 6040 Acked Read 10 Sent Done Done false
19836 6041 Acked Write 10 Sent Done Done false
19837 6042 Acked Read 10 Sent Done Done false
19838 6043 Acked Read 2 Sent Done Done false
19839 6044 Acked Write 4 Sent Done Done false
19840 6045 Acked Read 5 Sent Done Done false
19841 6046 Acked Write 5 Sent Done Done false
19842 6047 Acked Read 4 Sent Done Done false
19843 6048 Acked Write 8 Sent Done Done false
19844 6049 Acked Read 8 Sent Done Done false
19845 6050 Acked Read 6 Sent Done Done false
19846 6051 Acked Read 1 Sent Done Done false
19847 6052 Acked Write 9 Sent Done Done false
19848 6053 Acked Write 4 Sent Done Done false
19849 6054 Acked Read 2 Sent Done Done false
19850 6055 Acked Read 2 Sent Done Done false
19851 6056 Acked Write 1 Sent Done Done false
19852 6057 Acked Read 3 Sent Done Done false
19853 6058 Acked Read 7 Sent Done Done false
19854 6059 Acked Read 5 Sent Done Done false
19855 6060 Acked Write 4 Sent Done Done false
19856 6061 Acked Write 3 Sent Done Done false
19857 6062 Acked Read 1 Sent Done Done false
19858 6063 Acked Write 8 Sent Done Done false
19859 6064 Acked Flush 0 Sent Done Done false
19860 6065 Acked Read 9 Sent Done Done false
19861 6066 Acked Read 1 Sent Done Done false
19862 6067 Acked Read 4 Sent Done Done false
19863 6068 Acked Read 8 Sent Done Done false
19864 6069 Acked Write 1 Sent Done Done false
19865 6070 Acked Write 2 Sent Done Done false
19866 6071 Acked Read 9 Sent Done Done false
19867 6072 Acked Write 5 Sent Done Done false
19868 6073 Acked Read 7 Sent Done Done false
19869 6074 Acked Read 4 Sent Done Done false
19870 6075 Acked Write 1 Sent Done Done false
19871 6076 Acked Read 3 Sent Done Done false
19872 6077 Acked Write 3 Sent Done Done false
19873 6078 Acked Read 5 Sent Done Done false
19874 6079 Acked Read 4 Sent Done Done false
19875 6080 Acked Write 10 Sent Done Done false
19876 6081 Acked Read 9 Sent Done Done false
19877 6082 Acked Write 5 Sent Done Done false
19878 6083 Acked Write 1 Sent Done Done false
19879 6084 Acked Write 10 Sent Done Done false
19880 6085 Acked Flush 0 Sent Done Done false
19881 6086 Acked Read 3 Sent Done Done false
19882 6087 Acked Flush 0 Sent Done Done false
19883 6088 Acked Write 9 Sent Done Done false
19884 6089 Acked Read 1 Sent Done Done false
19885 6090 Acked Write 10 Sent Done Done false
19886 6091 Acked Read 1 Sent Done Done false
19887 6092 Acked Write 5 Sent Done Done false
19888 6093 Acked Flush 0 Sent Done Done false
19889 6094 Acked Read 7 Sent Done Done false
19890 6095 Acked Read 6 Sent Done Done false
19891 6096 Acked Write 5 Sent Done Done false
19892 6097 Acked Read 1 Sent Done Done false
19893 6098 Acked Write 9 Sent Done Done false
19894 6099 Acked Write 2 Sent Done Done false
19895 6100 Acked Read 8 Sent Done Done false
19896 6101 Acked Write 7 Sent Done Done false
19897 6102 Acked Read 3 Sent Done Done false
19898 6103 Acked Write 3 Sent Done Done false
19899 6104 Acked Read 10 Sent Done Done false
19900 6105 Acked Read 5 Sent Done Done false
19901 6106 Acked Read 2 Sent Done Done false
19902 6107 Acked Write 3 Sent Done Done false
19903 6108 Acked Read 10 Sent Done Done false
19904 6109 Acked Read 10 Sent Done Done false
19905 6110 Acked Read 4 Sent Done Done false
19906 6111 Acked Read 10 Sent Done Done false
19907 6112 Acked Read 1 Sent Done Done false
19908 6113 Acked Read 1 Sent Done Done false
19909 6114 Acked Write 6 Sent Done Done false
19910 6115 Acked Read 1 Sent Done Done false
19911 6116 Acked Write 6 Sent Done Done false
19912 6117 Acked Write 10 Sent Done Done false
19913 6118 Acked Write 4 Sent Done Done false
19914 6119 Acked Write 6 Sent Done Done false
19915 6120 Acked Write 4 Sent Done Done false
19916 6121 Acked Write 1 Sent Done Done false
19917 6122 Acked Read 8 Sent Done Done false
19918 6123 Acked Write 5 Sent Done Done false
19919 6124 Acked Write 9 Sent Done Done false
19920 6125 Acked Read 5 Sent Done Done false
19921 6126 Acked Write 7 Sent Done Done false
19922 6127 Acked Write 1 Sent Done Done false
19923 6128 Acked Read 4 Sent Done Done false
19924 6129 Acked Read 10 Sent Done Done false
19925 6130 Acked Read 7 Sent Done Done false
19926 6131 Acked Read 6 Sent Done Done false
19927 6132 Acked Flush 0 Sent Done Done false
19928 6133 Acked Write 6 Sent Done Done false
19929 6134 Acked Write 1 Sent Done Done false
19930 6135 Acked Read 5 Sent Done Done false
19931 6136 Acked Read 7 Sent Done Done false
19932 6137 Acked Write 6 Sent Done Done false
19933 6138 Acked Read 5 Sent Done Done false
19934 6139 Acked Flush 0 Sent Done Done false
19935 STATES DS:0 DS:1 DS:2 TOTAL
19936 Sent 302 0 0 302
19937 Done 0 302 302 604
19938 Skipped 0 0 0 0
19939 Error 0 0 0 0
19940 Last Flush: 5837 6139 6139
19941 Downstairs last five completed: 5837 5836 5835 5834 5833
19942 Upstairs last five completed: 6139 6138 6137 6136 6135
19943 CLIENT: Up:0 ds:302 act:2
199442024-11-07T14:07:28.561ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
199452024-11-07T14:07:28.561ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57256) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
199462024-11-07T14:07:28.562ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
199472024-11-07T14:07:28.563ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
199482024-11-07T14:07:28.563ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
199492024-11-07T14:07:28.563ZINFOcrucible: send last flush ID to this DS: 5837 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
199502024-11-07T14:07:28.563ZINFOcrucible: Replied this last flush ID: 5837 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
199512024-11-07T14:07:28.563ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
199522024-11-07T14:07:28.563ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
199532024-11-07T14:07:28.563ZINFOcrucible: [0] client re-new 302 jobs since flush 5837 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
199542024-11-07T14:07:28.563ZINFOcrucible: [0] Replayed 302 jobs since flush: 5837 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
19955 ----------------------------------------------------------------
19956 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19957 Downstairs last five completed: 6139 6138 6137 6136 6135
19958 Upstairs last five completed: 6139 6138 6137 6136 6135
19959 CLIENT: Up:0 ds:0 act:3
19960 Replay: All jobs finished
19961 Main task has work to do, go find it
19962 got dsc Stop(2)
19963 stop 2
19964 Nov 07 14:07:30.842 INFO request completed, latency_us: 362, [8830] Got stop action so:false kr:false
19965 response_code: 204, uri: /stop/cid/2, method: POST, req_id: eebf5a85-8fc7-408f-b289-16c4c2486ecc, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
199662024-11-07T14:07:30.843ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
19967 Nov 07 14:07:30.843 INFO{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-07T14:07:30.843408308Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"":"io task","client":"2","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
199682024-11-07T14:07:30.843ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
19969 request completed, latency_us: 138, {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:07:30.843465323Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"client":"2","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
19970 response_code: 200, uri: /state/cid/2, method: GET, req_id: {"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:07:30.843480762Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"client":"2","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
19971 e0704e4e-040a-47c8-bd39-51914ca21a4e, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
199722024-11-07T14:07:30.843ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
19973 [8830] Exited with: ExitStatus(unix_wait_status(9))
19974 [8830][2] reports Exit
19975 Nov 07 14:07:34.845 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2218eb07-88f7-41be-b632-7120ad20acd8, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
19976 001/300 Read block 4792 len 4096
19977 002/300 Read block 3758 len 28672
19978 003/300 Write block 3318 len 24576 data: 4 3 3 4 4 6
19979 004/300 Write block 2057 len 20480 data: 4 3 4 4 4
19980 005/300 Write block 994 len 8192 data: 3 3
19981 006/300 Read block 4163 len 32768
19982 007/300 Flush
19983 008/300 Flush
19984 009/300 Read block 4446 len 32768
19985 010/300 Flush
19986 011/300 Write block 3095 len 8192 data: 2 2
19987 012/300 Read block 2800 len 4096
19988 013/300 Write block 2143 len 16384 data: 2 4 2 3
19989 014/300 Write block 550 len 32768 data: 4 4 4 3 3 2 2 3
19990 015/300 Read block 1078 len 20480
19991 016/300 Write block 3005 len 32768 data: 3 3 3 3 3 3 5 5
19992 017/300 Read block 3474 len 16384
19993 018/300 Write block 4753 len 28672 data: 5 5 7 6 6 6 6
19994 019/300 Flush
19995 020/300 Read block 3404 len 36864
19996 021/300 Write block 3803 len 24576 data: 5 5 3 2 2 2
19997 022/300 Flush
19998 023/300 Write block 2974 len 24576 data: 5 4 6 6 6 5
19999 024/300 Read block 2445 len 20480
20000 025/300 Flush
20001 026/300 Flush
20002 027/300 Read block 2623 len 16384
20003 028/300 Flush
20004 029/300 Write block 1507 len 32768 data: 6 5 6 4 4 4 3 2
20005 030/300 Read block 2768 len 16384
20006 031/300 Read block 1116 len 12288
20007 032/300 Write block 1732 len 32768 data: 2 2 2 3 3 3 4 4
20008 033/300 Write block 3747 len 36864 data: 4 5 5 5 6 6 7 6 4
20009 034/300 Write block 2787 len 4096 data: 6
20010 035/300 Flush
20011 036/300 Write block 4694 len 32768 data: 4 4 5 8 6 5 5 6
20012 037/300 Flush
20013 038/300 Read block 230 len 16384
20014 039/300 Read block 1259 len 24576
20015 040/300 Read block 4944 len 36864
20016 041/300 Flush
20017 042/300 Write block 2079 len 40960 data: 4 5 5 5 5 3 3 3 2 3
20018 043/300 Write block 1481 len 20480 data: 6 4 4 4 4
20019 044/300 Write block 4919 len 32768 data: 8 8 5 5 4 5 5 6
20020 045/300 Read block 4950 len 16384
20021 046/300 Read block 93 len 28672
20022 047/300 Read block 1287 len 12288
20023 048/300 Write block 2708 len 40960 data: 3 4 5 4 2 2 2 2 2 2
20024 049/300 Read block 412 len 16384
20025 050/300 Read block 4399 len 20480
20026 051/300 Write block 4405 len 40960 data: 2 2 2 2 2 2 2 2 3 3
20027 052/300 Write block 3438 len 20480 data: 3 5 5 6 6
20028 053/300 Read block 4385 len 32768
20029 054/300 Write block 878 len 40960 data: 5 4 3 3 3 4 5 5 4 5
20030 055/300 Read block 4922 len 28672
20031 056/300 Read block 4722 len 36864
20032 057/300 Read block 3104 len 32768
20033 058/300 Read block 1371 len 8192
20034 059/300 Write block 2912 len 32768 data: 7 6 6 5 5 4 4 4
20035 060/300 Write block 4542 len 12288 data: 5 5 5
20036 061/300 Read block 1010 len 28672
20037 062/300 Read block 683 len 8192
20038 063/300 Write block 3569 len 36864 data: 5 4 3 3 3 3 3 3 4
20039 064/300 Write block 707 len 28672 data: 8 7 6 5 4 4 4
20040 065/300 Flush
20041 066/300 Write block 4362 len 8192 data: 5 4
20042 067/300 Write block 2526 len 36864 data: 3 3 4 4 4 3 3 2 3
20043 068/300 Read block 3953 len 16384
20044 069/300 Write block 4233 len 32768 data: 4 5 5 4 5 6 7 6
20045 070/300 Write block 1168 len 8192 data: 3 3
20046 071/300 Read block 652 len 12288
20047 072/300 Read block 4748 len 4096
20048 073/300 Flush
20049 074/300 Write block 1172 len 36864 data: 5 5 5 4 4 4 4 3 3
20050 075/300 Write block 287 len 12288 data: 5 5 6
20051 076/300 Read block 3104 len 40960
20052 077/300 Flush
20053 078/300 Write block 898 len 40960 data: 8 9 9 8 7 9 8 6 6 5
20054 079/300 Write block 4177 len 28672 data: 5 6 5 4 4 4 4
20055 080/300 Read block 1911 len 8192
20056 081/300 Read block 4048 len 28672
20057 082/300 Read block 3695 len 24576
20058 083/300 Read block 578 len 16384
20059 084/300 Write block 4328 len 4096 data: 2
20060 085/300 Write block 3455 len 8192 data: 2 3
20061 086/300 Write block 802 len 4096 data: 3
20062 087/300 Write block 2143 len 36864 data: 3 5 3 4 2 2 2 3 3
20063 088/300 Read block 2246 len 24576
20064 089/300 Write block 1156 len 40960 data: 5 4 4 5 4 4 3 3 3 3
20065 090/300 Read block 3226 len 28672
20066 091/300 Write block 62 len 20480 data: 3 2 2 2 2
20067 092/300 Read block 3739 len 36864
20068 093/300 Write block 4535 len 4096 data: 6
20069 094/300 Read block 1117 len 8192
20070 095/300 Write block 1205 len 12288 data: 3 4 3
20071 096/300 Read block 11 len 40960
20072 097/300 Write block 2499 len 16384 data: 5 4 3 3
20073 098/300 Read block 559 len 8192
20074 099/300 Write block 523 len 36864 data: 3 2 2 2 6 5 5 4 6
20075 100/300 Read block 3277 len 12288
20076 101/300 Write block 1876 len 16384 data: 5 6 6 6
20077 102/300 Read block 2338 len 24576
20078 103/300 Write block 0 len 32768 data: 2 3 4 6 6 5 5 4
20079 104/300 Write block 1923 len 4096 data: 3
20080 105/300 Write block 2463 len 12288 data: 3 3 3
20081 106/300 Read block 4560 len 4096
20082 107/300 Read block 687 len 20480
20083 108/300 Read block 3800 len 16384
20084 109/300 Read block 2005 len 36864
20085 110/300 Write block 4981 len 12288 data: 3 3 3
20086 111/300 Flush
20087 112/300 Read block 3276 len 20480
20088 113/300 Read block 2424 len 8192
20089 114/300 Write block 3402 len 28672 data: 7 7 7 5 5 3 3
20090 115/300 Write block 353 len 40960 data: 3 2 2 2 2 2 2 3 3 4
20091 116/300 Read block 2985 len 16384
20092 117/300 Write block 2658 len 32768 data: 5 5 4 4 6 6 6 6
20093 118/300 Read block 4337 len 28672
20094 119/300 Read block 1125 len 28672
20095 120/300 Read block 3993 len 8192
20096 121/300 Read block 2578 len 16384
20097 122/300 Read block 2885 len 28672
20098 123/300 Flush
20099 124/300 Read block 2019 len 4096
20100 125/300 Read block 3410 len 4096
20101 126/300 Read block 2061 len 16384
20102 127/300 Write block 2753 len 12288 data: 4 4 4
20103 128/300 Read block 907 len 12288
20104 129/300 Write block 4217 len 20480 data: 2 3 2 4 4
20105 130/300 Read block 3096 len 20480
20106 131/300 Read block 4170 len 28672
20107 132/300 Write block 3829 len 40960 data: 6 5 5 4 5 5 5 5 4 4
20108 133/300 Flush
20109 134/300 Write block 1711 len 32768 data: 5 4 4 4 5 4 4 4
20110 135/300 Read block 2192 len 8192
20111 136/300 Read block 3148 len 8192
20112 137/300 Write block 3193 len 24576 data: 2 3 3 4 2 4
20113 138/300 Read block 3229 len 20480
20114 139/300 Read block 3829 len 36864
20115 140/300 Read block 4631 len 32768
20116 141/300 Write block 4402 len 28672 data: 3 3 3 3 3 3 3
20117 142/300 Read block 164 len 16384
20118 143/300 Write block 4381 len 8192 data: 3 4
20119 144/300 Flush
20120 145/300 Read block 3793 len 8192
20121 146/300 Read block 4647 len 40960
20122 147/300 Flush
20123 148/300 Read block 127 len 4096
20124 149/300 Write block 1504 len 24576 data: 4 5 4 7 6 7
20125 150/300 Read block 1362 len 36864
20126 151/300 Read block 1344 len 24576
20127 152/300 Flush
20128 153/300 Read block 2469 len 16384
20129 154/300 Read block 4405 len 12288
20130 155/300 Write block 2177 len 12288 data: 2 2 2
20131 156/300 Flush
20132 157/300 Write block 3821 len 16384 data: 3 4 4 3
20133 158/300 Write block 692 len 40960 data: 3 3 3 4 4 5 2 2 2 2
20134 159/300 Write block 1620 len 12288 data: 3 3 2
20135 160/300 Write block 1270 len 28672 data: 3 3 3 3 3 3 2
20136 161/300 Read block 4962 len 4096
20137 162/300 Read block 1588 len 36864
20138 163/300 Read block 1596 len 40960
20139 164/300 Write block 2495 len 4096 data: 7
20140 165/300 Read block 4183 len 40960
20141 166/300 Read block 4892 len 4096
20142 167/300 Flush
20143 168/300 Read block 2449 len 40960
20144 169/300 Write block 4156 len 8192 data: 2 2
20145 170/300 Read block 1268 len 4096
20146 171/300 Read block 25 len 12288
20147 172/300 Flush
20148 173/300 Write block 2419 len 24576 data: 5 5 5 4 5 5
20149 174/300 Read block 21 len 28672
20150 175/300 Read block 2903 len 36864
20151 176/300 Read block 4729 len 16384
20152 177/300 Write block 4306 len 16384 data: 4 4 3 5
20153 178/300 Write block 957 len 12288 data: 6 4 3
20154 179/300 Read block 1323 len 28672
20155 180/300 Read block 3709 len 40960
20156 181/300 Read block 1258 len 16384
20157 182/300 Read block 1417 len 4096
20158 183/300 Read block 2718 len 24576
20159 184/300 Write block 4323 len 12288 data: 2 2 2
20160 185/300 Read block 706 len 36864
20161 186/300 Write block 4216 len 8192 data: 3 3
20162 187/300 Write block 1126 len 40960 data: 5 4 3 4 4 3 3 3 5 5
20163 188/300 Write block 4460 len 36864 data: 2 3 4 3 5 5 4 4 4
20164 189/300 Write block 4163 len 28672 data: 3 4 3 3 4 4 5
20165 190/300 Flush
20166 191/300 Write block 2028 len 36864 data: 4 4 3 3 4 4 3 2 3
20167 192/300 Write block 2243 len 24576 data: 3 3 3 3 3 3
20168 193/300 Write block 2599 len 36864 data: 5 3 3 3 4 3 2 2 2
20169 194/300 Read block 4603 len 12288
20170 195/300 Read block 939 len 16384
20171 196/300 Write block 4670 len 24576 data: 3 3 5 4 4 4
20172 197/300 Read block 4144 len 28672
20173 198/300 Read block 1154 len 4096
20174 199/300 Read block 156 len 4096
20175 200/300 Write block 922 len 28672 data: 2 2 2 2 2 3 4
20176 201/300 Write block 2664 len 16384 data: 7 7 5 4
20177 202/300 Read block 2145 len 20480
20178 203/300 Read block 2999 len 4096
20179 204/300 Write block 224 len 28672 data: 4 4 8 9 4 3 4
20180 205/300 Read block 4535 len 16384
20181 206/300 Read block 3624 len 12288
20182 207/300 Flush
20183 208/300 Write block 1278 len 28672 data: 4 4 4 4 5 5 5
20184 209/300 Read block 1091 len 36864
20185 210/300 Read block 397 len 8192
20186 211/300 Write block 4602 len 28672 data: 6 5 6 5 3 3 4
20187 212/300 Flush
20188 213/300 Flush
20189 214/300 Read block 663 len 8192
20190 215/300 Read block 4812 len 40960
20191 216/300 Write block 2852 len 12288 data: 5 6 7
20192 217/300 Flush
20193 218/300 Write block 1515 len 20480 data: 2 2 2 2 2
20194 219/300 Flush
20195 220/300 Write block 3161 len 28672 data: 3 2 2 2 2 2 3
20196 221/300 Write block 3379 len 16384 data: 6 6 6 5
20197 222/300 Write block 1690 len 32768 data: 2 2 2 2 2 2 2 2
20198 223/300 Read block 3905 len 40960
20199 224/300 Read block 3431 len 20480
20200 225/300 Read block 2123 len 36864
20201 226/300 Read block 3785 len 8192
20202 227/300 Read block 4159 len 40960
20203 228/300 Write block 1487 len 32768 data: 3 3 3 3 3 4 5 5
20204 229/300 Flush
20205 230/300 Read block 3183 len 16384
20206 231/300 Write block 211 len 8192 data: 5 4
20207 232/300 Read block 3497 len 16384
20208 233/300 Read block 1116 len 24576
20209 234/300 Read block 503 len 28672
20210 235/300 Read block 3214 len 20480
20211 236/300 Write block 859 len 28672 data: 4 4 4 4 4 5 4
20212 237/300 Write block 1803 len 16384 data: 3 3 4 5
20213 238/300 Write block 2847 len 16384 data: 4 4 4 4
20214 239/300 Flush
20215 240/300 Write block 2034 len 4096 data: 4
20216 241/300 Read block 1306 len 20480
20217 242/300 Write block 1864 len 20480 data: 4 4 3 3 3
20218 243/300 Read block 4710 len 8192
20219 244/300 Write block 3795 len 28672 data: 6 5 6 6 6 6 5
20220 245/300 Flush
20221 246/300 Read block 4573 len 32768
20222 247/300 Read block 4734 len 4096
20223 248/300 Write block 1774 len 28672 data: 6 6 4 4 5 5 4
20224 249/300 Write block 3275 len 4096 data: 6
20225 250/300 Write block 321 len 4096 data: 2
20226 251/300 Read block 170 len 32768
20227 252/300 Write block 4103 len 24576 data: 5 5 6 6 4 4
20228 253/300 Read block 1027 len 32768
20229 254/300 Flush
20230 255/300 Read block 4581 len 8192
20231 256/300 Write block 172 len 40960 data: 3 3 3 3 3 2 3 5 5 5
20232 257/300 Read block 1559 len 40960
20233 258/300 Write block 54 len 16384 data: 4 5 3 3
20234 259/300 Read block 3894 len 12288
20235 260/300 Flush
20236 261/300 Read block 4202 len 40960
20237 262/300 Write block 4281 len 16384 data: 5 3 3 3
20238 263/300 Read block 4043 len 28672
20239 264/300 Read block 4623 len 12288
20240 265/300 Write block 876 len 28672 data: 4 5 6 5 4 4 4
20241 266/300 Write block 3494 len 8192 data: 5 5
20242 267/300 Read block 869 len 4096
20243 268/300 Read block 207 len 8192
20244 269/300 Flush
20245 270/300 Read block 3461 len 24576
20246 271/300 Write block 4503 len 12288 data: 4 5 3
20247 272/300 Flush
20248 273/300 Read block 4764 len 20480
20249 274/300 Read block 2116 len 8192
20250 275/300 Write block 1959 len 20480 data: 4 5 5 4 4
20251 276/300 Read block 4029 len 8192
20252 277/300 Read block 4623 len 20480
20253 278/300 Read block 40 len 24576
20254 279/300 Write block 631 len 36864 data: 4 5 3 3 5 5 2 2 2
20255 280/300 Write block 1428 len 24576 data: 3 4 4 5 5 5
20256 281/300 Flush
20257 282/300 Write block 2438 len 40960 data: 5 5 5 5 5 5 5 6 7 5
20258 283/300 Read block 3274 len 40960
20259 284/300 Read block 3740 len 8192
20260 285/300 Read block 4373 len 32768
20261 286/300 Write block 1613 len 20480 data: 3 3 3 3 3
20262 287/300 Read block 1923 len 36864
20263 288/300 Read block 2024 len 36864
20264 289/300 Read block 3134 len 28672
20265 290/300 Write block 4988 len 20480 data: 4 5 4 4 3
20266 291/300 Write block 1554 len 20480 data: 3 3 3 3 3
20267 292/300 Read block 4312 len 16384
20268 293/300 Write block 277 len 4096 data: 4
20269 294/300 Write block 1767 len 24576 data: 3 4 6 5 6 5
20270 295/300 Write block 3713 len 40960 data: 5 5 6 6 7 8 8 8 7 7
20271 296/300 Write block 2646 len 16384 data: 5 5 6 6
20272 297/300 Read block 3935 len 32768
20273 298/300 Read block 1548 len 24576
20274 299/300 Read block 3287 len 12288
20275 300/300 Write block 1153 len 32768 data: 3 5 5 6 5 5 6 5
20276 301/300 Read block 2583 len 40960
20277 Main task has work to do, go find it
20278 got dsc Start(2)
20279 start 2
20280 Nov 07 14:07:35.133 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/2, method: POST, req_id: 6e4805ac-4ab8-446b-9674-c163251280fc, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
20281 Replay: started 2, returned:Ok(())
20282 ----------------------------------------------------------------
20283 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20284 States: Active Active Offline
20285 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20286 6140 Acked Read 1 Done Done Sent false
20287 6141 Acked Read 7 Done Done Sent false
20288 6142 Acked Write 6 Done Done Sent false
20289 6143 Acked Write 5 Done Done Sent false
20290 6144 Acked Write 2 Done Done Sent false
20291 6145 Acked Read 8 Done Done Sent false
20292 6146 Acked Flush 0 Done Done Sent false
20293 6147 Acked Flush 0 Done Done Sent false
20294 6148 Acked Read 8 Done Done Sent false
20295 6149 Acked Flush 0 Done Done Sent false
20296 6150 Acked Write 2 Done Done Sent false
20297 6151 Acked Read 1 Done Done Sent false
20298 6152 Acked Write 4 Done Done Sent false
20299 6153 Acked Write 8 Done Done Sent false
20300 6154 Acked Read 5 Done Done Sent false
20301 6155 Acked Write 8 Done Done Sent false
20302 6156 Acked Read 4 Done Done Sent false
20303 6157 Acked Write 7 Done Done Sent false
20304 6158 Acked Flush 0 Done Done Sent false
20305 6159 Acked Read 9 Done Done Sent false
20306 6160 Acked Write 6 Done Done Sent false
20307 6161 Acked Flush 0 Done Done Sent false
20308 6162 Acked Write 6 Done Done Sent false
20309 6163 Acked Read 5 Done Done Sent false
20310 6164 Acked Flush 0 Done Done Sent false
20311 6165 Acked Flush 0 Done Done Sent false
20312 6166 Acked Read 4 Done Done Sent false
20313 6167 Acked Flush 0 Done Done Sent false
20314 6168 Acked Write 8 Done Done Sent false
20315 6169 Acked Read 4 Done Done Sent false
20316 6170 Acked Read 3 Done Done Sent false
20317 6171 Acked Write 8 Done Done Sent false
20318 6172 Acked Write 9 Done Done Sent false
20319 6173 Acked Write 1 Done Done Sent false
20320 6174 Acked Flush 0 Done Done Sent false
20321 6175 Acked Write 8 Done Done Sent false
20322 6176 Acked Flush 0 Done Done Sent false
20323 6177 Acked Read 4 Done Done Sent false
20324 6178 Acked Read 6 Done Done Sent false
20325 6179 Acked Read 9 Done Done Sent false
20326 6180 Acked Flush 0 Done Done Sent false
20327 6181 Acked Write 10 Done Done Sent false
20328 6182 Acked Write 5 Done Done Sent false
20329 6183 Acked Write 8 Done Done Sent false
20330 6184 Acked Read 4 Done Done Sent false
20331 6185 Acked Read 7 Done Done Sent false
20332 6186 Acked Read 3 Done Done Sent false
20333 6187 Acked Write 10 Done Done Sent false
20334 6188 Acked Read 4 Done Done Sent false
20335 6189 Acked Read 5 Done Done Sent false
20336 6190 Acked Write 10 Done Done Sent false
20337 6191 Acked Write 5 Done Done Sent false
20338 6192 Acked Read 8 Done Done Sent false
20339 6193 Acked Write 10 Done Done Sent false
20340 6194 Acked Read 7 Done Done Sent false
20341 6195 Acked Read 9 Done Done Sent false
20342 6196 Acked Read 8 Done Done Sent false
20343 6197 Acked Read 2 Done Done Sent false
20344 6198 Acked Write 8 Done Done Sent false
20345 6199 Acked Write 3 Done Done Sent false
20346 6200 Acked Read 7 Done Done Sent false
20347 6201 Acked Read 2 Done Done Sent false
20348 6202 Acked Write 9 Done Done Sent false
20349 6203 Acked Write 7 Done Done Sent false
20350 6204 Acked Flush 0 Done Done Sent false
20351 6205 Acked Write 2 Done Done Sent false
20352 6206 Acked Write 9 Done Done Sent false
20353 6207 Acked Read 4 Done Done Sent false
20354 6208 Acked Write 8 Done Done Sent false
20355 6209 Acked Write 2 Done Done Sent false
20356 6210 Acked Read 3 Done Done Sent false
20357 6211 Acked Read 1 Done Done Sent false
20358 6212 Acked Flush 0 Done Done Sent false
20359 6213 Acked Write 9 Done Done Sent false
20360 6214 Acked Write 3 Done Done Sent false
20361 6215 Acked Read 10 Done Done Sent false
20362 6216 Acked Flush 0 Done Done Sent false
20363 6217 Acked Write 10 Done Done Sent false
20364 6218 Acked Write 7 Done Done Sent false
20365 6219 Acked Read 2 Done Done Sent false
20366 6220 Acked Read 7 Done Done Sent false
20367 6221 Acked Read 6 Done Done Sent false
20368 6222 Acked Read 4 Done Done Sent false
20369 6223 Acked Write 1 Done Done Sent false
20370 6224 Acked Write 2 Done Done Sent false
20371 6225 Acked Write 1 Done Done Sent false
20372 6226 Acked Write 9 Done Done Sent false
20373 6227 Acked Read 6 Done Done Sent false
20374 6228 Acked Write 10 Done Done Sent false
20375 6229 Acked Read 7 Done Done Sent false
20376 6230 Acked Write 5 Done Done Sent false
20377 6231 Acked Read 9 Done Done Sent false
20378 6232 Acked Write 1 Done Done Sent false
20379 6233 Acked Read 2 Done Done Sent false
20380 6234 Acked Write 3 Done Done Sent false
20381 6235 Acked Read 10 Done Done Sent false
20382 6236 Acked Write 4 Done Done Sent false
20383 6237 Acked Read 2 Done Done Sent false
20384 6238 Acked Write 9 Done Done Sent false
20385 6239 Acked Read 3 Done Done Sent false
20386 6240 Acked Write 4 Done Done Sent false
20387 6241 Acked Read 6 Done Done Sent false
20388 6242 Acked Write 8 Done Done Sent false
20389 6243 Acked Write 1 Done Done Sent false
20390 6244 Acked Write 3 Done Done Sent false
20391 6245 Acked Read 1 Done Done Sent false
20392 6246 Acked Read 5 Done Done Sent false
20393 6247 Acked Read 4 Done Done Sent false
20394 6248 Acked Read 9 Done Done Sent false
20395 6249 Acked Write 3 Done Done Sent false
20396 6250 Acked Flush 0 Done Done Sent false
20397 6251 Acked Read 5 Done Done Sent false
20398 6252 Acked Read 2 Done Done Sent false
20399 6253 Acked Write 7 Done Done Sent false
20400 6254 Acked Write 10 Done Done Sent false
20401 6255 Acked Read 4 Done Done Sent false
20402 6256 Acked Write 8 Done Done Sent false
20403 6257 Acked Read 7 Done Done Sent false
20404 6258 Acked Read 7 Done Done Sent false
20405 6259 Acked Read 2 Done Done Sent false
20406 6260 Acked Read 4 Done Done Sent false
20407 6261 Acked Read 7 Done Done Sent false
20408 6262 Acked Flush 0 Done Done Sent false
20409 6263 Acked Read 1 Done Done Sent false
20410 6264 Acked Read 1 Done Done Sent false
20411 6265 Acked Read 4 Done Done Sent false
20412 6266 Acked Write 3 Done Done Sent false
20413 6267 Acked Read 3 Done Done Sent false
20414 6268 Acked Write 5 Done Done Sent false
20415 6269 Acked Read 5 Done Done Sent false
20416 6270 Acked Read 7 Done Done Sent false
20417 6271 Acked Write 10 Done Done Sent false
20418 6272 Acked Flush 0 Done Done Sent false
20419 6273 Acked Write 8 Done Done Sent false
20420 6274 Acked Read 2 Done Done Sent false
20421 6275 Acked Read 2 Done Done Sent false
20422 6276 Acked Write 6 Done Done Sent false
20423 6277 Acked Read 5 Done Done Sent false
20424 6278 Acked Read 9 Done Done Sent false
20425 6279 Acked Read 8 Done Done Sent false
20426 6280 Acked Write 7 Done Done Sent false
20427 6281 Acked Read 4 Done Done Sent false
20428 6282 Acked Write 2 Done Done Sent false
20429 6283 Acked Flush 0 Done Done Sent false
20430 6284 Acked Read 2 Done Done Sent false
20431 6285 Acked Read 10 Done Done Sent false
20432 6286 Acked Flush 0 Done Done Sent false
20433 6287 Acked Read 1 Done Done Sent false
20434 6288 Acked Write 6 Done Done Sent false
20435 6289 Acked Read 9 Done Done Sent false
20436 6290 Acked Read 6 Done Done Sent false
20437 6291 Acked Flush 0 Done Done Sent false
20438 6292 Acked Read 4 Done Done Sent false
20439 6293 Acked Read 3 Done Done Sent false
20440 6294 Acked Write 3 Done Done Sent false
20441 6295 Acked Flush 0 Done Done Sent false
20442 6296 Acked Write 4 Done Done Sent false
20443 6297 Acked Write 10 Done Done Sent false
20444 6298 Acked Write 3 Done Done Sent false
20445 6299 Acked Write 7 Done Done Sent false
20446 6300 Acked Read 1 Done Done Sent false
20447 6301 Acked Read 9 Done Done Sent false
20448 6302 Acked Read 10 Done Done Sent false
20449 6303 Acked Write 1 Done Done Sent false
20450 6304 Acked Read 10 Done Done Sent false
20451 6305 Acked Read 1 Done Done Sent false
20452 6306 Acked Flush 0 Done Done Sent false
20453 6307 Acked Read 10 Done Done Sent false
20454 6308 Acked Write 2 Done Done Sent false
20455 6309 Acked Read 1 Done Done Sent false
20456 6310 Acked Read 3 Done Done Sent false
20457 6311 Acked Flush 0 Done Done Sent false
20458 6312 Acked Write 6 Done Done Sent false
20459 6313 Acked Read 7 Done Done Sent false
20460 6314 Acked Read 9 Done Done Sent false
20461 6315 Acked Read 4 Done Done Sent false
20462 6316 Acked Write 4 Done Done Sent false
20463 6317 Acked Write 3 Done Done Sent false
20464 6318 Acked Read 7 Done Done Sent false
20465 6319 Acked Read 10 Done Done Sent false
20466 6320 Acked Read 4 Done Done Sent false
20467 6321 Acked Read 1 Done Done Sent false
20468 6322 Acked Read 6 Done Done Sent false
20469 6323 Acked Write 3 Done Done Sent false
20470 6324 Acked Read 9 Done Done Sent false
20471 6325 Acked Write 2 Done Done Sent false
20472 6326 Acked Write 10 Done Done Sent false
20473 6327 Acked Write 9 Done Done Sent false
20474 6328 Acked Write 7 Done Done Sent false
20475 6329 Acked Flush 0 Done Done Sent false
20476 6330 Acked Write 9 Done Done Sent false
20477 6331 Acked Write 6 Done Done Sent false
20478 6332 Acked Write 9 Done Done Sent false
20479 6333 Acked Read 3 Done Done Sent false
20480 6334 Acked Read 4 Done Done Sent false
20481 6335 Acked Write 6 Done Done Sent false
20482 6336 Acked Read 7 Done Done Sent false
20483 6337 Acked Read 1 Done Done Sent false
20484 6338 Acked Read 1 Done Done Sent false
20485 6339 Acked Write 7 Done Done Sent false
20486 6340 Acked Write 4 Done Done Sent false
20487 6341 Acked Read 5 Done Done Sent false
20488 6342 Acked Read 1 Done Done Sent false
20489 6343 Acked Write 7 Done Done Sent false
20490 6344 Acked Read 4 Done Done Sent false
20491 6345 Acked Read 3 Done Done Sent false
20492 6346 Acked Flush 0 Done Done Sent false
20493 6347 Acked Write 7 Done Done Sent false
20494 6348 Acked Read 9 Done Done Sent false
20495 6349 Acked Read 2 Done Done Sent false
20496 6350 Acked Write 7 Done Done Sent false
20497 6351 Acked Flush 0 Done Done Sent false
20498 6352 Acked Flush 0 Done Done Sent false
20499 6353 Acked Read 2 Done Done Sent false
20500 6354 Acked Read 10 Done Done Sent false
20501 6355 Acked Write 3 Done Done Sent false
20502 6356 Acked Flush 0 Done Done Sent false
20503 6357 Acked Write 5 Done Done Sent false
20504 6358 Acked Flush 0 Done Done Sent false
20505 6359 Acked Write 7 Done Done Sent false
20506 6360 Acked Write 4 Done Done Sent false
20507 6361 Acked Write 8 Done Done Sent false
20508 6362 Acked Read 10 Done Done Sent false
20509 6363 Acked Read 5 Done Done Sent false
20510 6364 Acked Read 9 Done Done Sent false
20511 6365 Acked Read 2 Done Done Sent false
20512 6366 Acked Read 10 Done Done Sent false
20513 6367 Acked Write 8 Done Done Sent false
20514 6368 Acked Flush 0 Done Done Sent false
20515 6369 Acked Read 4 Done Done Sent false
20516 6370 Acked Write 2 Done Done Sent false
20517 6371 Acked Read 4 Done Done Sent false
20518 6372 Acked Read 6 Done Done Sent false
20519 6373 Acked Read 7 Done Done Sent false
20520 6374 Acked Read 5 Done Done Sent false
20521 6375 Acked Write 7 Done Done Sent false
20522 6376 Acked Write 4 Done Done Sent false
20523 6377 Acked Write 4 Done Done Sent false
20524 6378 Acked Flush 0 Done Done Sent false
20525 6379 Acked Write 1 Done Done Sent false
20526 6380 Acked Read 5 Done Done Sent false
20527 6381 Acked Write 5 Done Done Sent false
20528 6382 Acked Read 2 Done Done Sent false
20529 6383 Acked Write 7 Done Done Sent false
20530 6384 Acked Flush 0 Done Done Sent false
20531 6385 Acked Read 8 Done Done Sent false
20532 6386 Acked Read 1 Done Done Sent false
20533 6387 Acked Write 7 Done Done Sent false
20534 6388 Acked Write 1 Done Done Sent false
20535 6389 Acked Write 1 Done Done Sent false
20536 6390 Acked Read 8 Done Done Sent false
20537 6391 Acked Write 6 Done Done Sent false
20538 6392 Acked Read 8 Done Done Sent false
20539 6393 Acked Flush 0 Done Done Sent false
20540 6394 Acked Read 2 Done Done Sent false
20541 6395 Acked Write 10 Done Done Sent false
20542 6396 Acked Read 10 Done Done Sent false
20543 6397 Acked Write 4 Done Done Sent false
20544 6398 Acked Read 3 Done Done Sent false
20545 6399 Acked Flush 0 Done Done Sent false
20546 6400 Acked Read 10 Done Done Sent false
20547 6401 Acked Write 4 Done Done Sent false
20548 6402 Acked Read 7 Done Done Sent false
20549 6403 Acked Read 3 Done Done Sent false
20550 6404 Acked Write 7 Done Done Sent false
20551 6405 Acked Write 2 Done Done Sent false
20552 6406 Acked Read 1 Done Done Sent false
20553 6407 Acked Read 2 Done Done Sent false
20554 6408 Acked Flush 0 Done Done Sent false
20555 6409 Acked Read 6 Done Done Sent false
20556 6410 Acked Write 3 Done Done Sent false
20557 6411 Acked Flush 0 Done Done Sent false
20558 6412 Acked Read 5 Done Done Sent false
20559 6413 Acked Read 2 Done Done Sent false
20560 6414 Acked Write 5 Done Done Sent false
20561 6415 Acked Read 2 Done Done Sent false
20562 6416 Acked Read 5 Done Done Sent false
20563 6417 Acked Read 6 Done Done Sent false
20564 6418 Acked Write 9 Done Done Sent false
20565 6419 Acked Write 6 Done Done Sent false
20566 6420 Acked Flush 0 Done Done Sent false
20567 6421 Acked Write 10 Done Done Sent false
20568 6422 Acked Read 10 Done Done Sent false
20569 6423 Acked Read 2 Done Done Sent false
20570 6424 Acked Read 8 Done Done Sent false
20571 6425 Acked Write 5 Done Done Sent false
20572 6426 Acked Read 9 Done Done Sent false
20573 6427 Acked Read 9 Done Done Sent false
20574 6428 Acked Read 7 Done Done Sent false
20575 6429 Acked Write 5 Done Done Sent false
20576 6430 Acked Write 5 Done Done Sent false
20577 6431 Acked Read 4 Done Done Sent false
20578 6432 Acked Write 1 Done Done Sent false
20579 6433 Acked Write 6 Done Done Sent false
20580 6434 Acked Write 10 Done Done Sent false
20581 6435 Acked Write 4 Done Done Sent false
20582 6436 Acked Read 8 Done Done Sent false
20583 6437 Acked Read 6 Done Done Sent false
20584 6438 Acked Read 3 Done Done Sent false
20585 6439 Acked Write 8 Done Done Sent false
20586 6440 Acked Read 10 Done Done Sent false
20587 STATES DS:0 DS:1 DS:2 TOTAL
20588 Sent 0 0 301 301
20589 Done 301 301 0 602
20590 Skipped 0 0 0 0
20591 Error 0 0 0 0
20592 Last Flush: 6420 6420 6139
20593 Downstairs last five completed: 6139 6138 6137 6136 6135
20594 Upstairs last five completed: 6440 6439 6438 6437 6436
20595 CLIENT: Up:0 ds:301 act:2
20596 ----------------------------------------------------------------
20597 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20598 States: Active Active Offline
20599 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20600 6140 Acked Read 1 Done Done Sent false
20601 6141 Acked Read 7 Done Done Sent false
20602 6142 Acked Write 6 Done Done Sent false
20603 6143 Acked Write 5 Done Done Sent false
20604 6144 Acked Write 2 Done Done Sent false
20605 6145 Acked Read 8 Done Done Sent false
20606 6146 Acked Flush 0 Done Done Sent false
20607 6147 Acked Flush 0 Done Done Sent false
20608 6148 Acked Read 8 Done Done Sent false
20609 6149 Acked Flush 0 Done Done Sent false
20610 6150 Acked Write 2 Done Done Sent false
20611 6151 Acked Read 1 Done Done Sent false
20612 6152 Acked Write 4 Done Done Sent false
20613 6153 Acked Write 8 Done Done Sent false
20614 6154 Acked Read 5 Done Done Sent false
20615 6155 Acked Write 8 Done Done Sent false
20616 6156 Acked Read 4 Done Done Sent false
20617 6157 Acked Write 7 Done Done Sent false
20618 6158 Acked Flush 0 Done Done Sent false
20619 6159 Acked Read 9 Done Done Sent false
20620 6160 Acked Write 6 Done Done Sent false
20621 6161 Acked Flush 0 Done Done Sent false
20622 6162 Acked Write 6 Done Done Sent false
20623 6163 Acked Read 5 Done Done Sent false
20624 6164 Acked Flush 0 Done Done Sent false
20625 6165 Acked Flush 0 Done Done Sent false
20626 6166 Acked Read 4 Done Done Sent false
20627 6167 Acked Flush 0 Done Done Sent false
20628 6168 Acked Write 8 Done Done Sent false
20629 6169 Acked Read 4 Done Done Sent false
20630 6170 Acked Read 3 Done Done Sent false
20631 6171 Acked Write 8 Done Done Sent false
20632 6172 Acked Write 9 Done Done Sent false
20633 6173 Acked Write 1 Done Done Sent false
20634 6174 Acked Flush 0 Done Done Sent false
20635 6175 Acked Write 8 Done Done Sent false
20636 6176 Acked Flush 0 Done Done Sent false
20637 6177 Acked Read 4 Done Done Sent false
20638 6178 Acked Read 6 Done Done Sent false
20639 6179 Acked Read 9 Done Done Sent false
20640 6180 Acked Flush 0 Done Done Sent false
20641 6181 Acked Write 10 Done Done Sent false
20642 6182 Acked Write 5 Done Done Sent false
20643 6183 Acked Write 8 Done Done Sent false
20644 6184 Acked Read 4 Done Done Sent false
20645 6185 Acked Read 7 Done Done Sent false
20646 6186 Acked Read 3 Done Done Sent false
20647 6187 Acked Write 10 Done Done Sent false
20648 6188 Acked Read 4 Done Done Sent false
20649 6189 Acked Read 5 Done Done Sent false
20650 6190 Acked Write 10 Done Done Sent false
20651 6191 Acked Write 5 Done Done Sent false
20652 6192 Acked Read 8 Done Done Sent false
20653 6193 Acked Write 10 Done Done Sent false
20654 6194 Acked Read 7 Done Done Sent false
20655 6195 Acked Read 9 Done Done Sent false
20656 6196 Acked Read 8 Done Done Sent false
20657 6197 Acked Read 2 Done Done Sent false
20658 6198 Acked Write 8 Done Done Sent false
20659 6199 Acked Write 3 Done Done Sent false
20660 6200 Acked Read 7 Done Done Sent false
20661 6201 Acked Read 2 Done Done Sent false
20662 6202 Acked Write 9 Done Done Sent false
20663 6203 Acked Write 7 Done Done Sent false
20664 6204 Acked Flush 0 Done Done Sent false
20665 6205 Acked Write 2 Done Done Sent false
20666 6206 Acked Write 9 Done Done Sent false
20667 6207 Acked Read 4 Done Done Sent false
20668 6208 Acked Write 8 Done Done Sent false
20669 6209 Acked Write 2 Done Done Sent false
20670 6210 Acked Read 3 Done Done Sent false
20671 6211 Acked Read 1 Done Done Sent false
20672 6212 Acked Flush 0 Done Done Sent false
20673 6213 Acked Write 9 Done Done Sent false
20674 6214 Acked Write 3 Done Done Sent false
20675 6215 Acked Read 10 Done Done Sent false
20676 6216 Acked Flush 0 Done Done Sent false
20677 6217 Acked Write 10 Done Done Sent false
20678 6218 Acked Write 7 Done Done Sent false
20679 6219 Acked Read 2 Done Done Sent false
20680 6220 Acked Read 7 Done Done Sent false
20681 6221 Acked Read 6 Done Done Sent false
20682 6222 Acked Read 4 Done Done Sent false
20683 6223 Acked Write 1 Done Done Sent false
20684 6224 Acked Write 2 Done Done Sent false
20685 6225 Acked Write 1 Done Done Sent false
20686 6226 Acked Write 9 Done Done Sent false
20687 6227 Acked Read 6 Done Done Sent false
20688 6228 Acked Write 10 Done Done Sent false
20689 6229 Acked Read 7 Done Done Sent false
20690 6230 Acked Write 5 Done Done Sent false
20691 6231 Acked Read 9 Done Done Sent false
20692 6232 Acked Write 1 Done Done Sent false
20693 6233 Acked Read 2 Done Done Sent false
20694 6234 Acked Write 3 Done Done Sent false
20695 6235 Acked Read 10 Done Done Sent false
20696 6236 Acked Write 4 Done Done Sent false
20697 6237 Acked Read 2 Done Done Sent false
20698 6238 Acked Write 9 Done Done Sent false
20699 6239 Acked Read 3 Done Done Sent false
20700 6240 Acked Write 4 Done Done Sent false
20701 6241 Acked Read 6 Done Done Sent false
20702 6242 Acked Write 8 Done Done Sent false
20703 6243 Acked Write 1 Done Done Sent false
20704 6244 Acked Write 3 Done Done Sent false
20705 6245 Acked Read 1 Done Done Sent false
20706 6246 Acked Read 5 Done Done Sent false
20707 6247 Acked Read 4 Done Done Sent false
20708 6248 Acked Read 9 Done Done Sent false
20709 6249 Acked Write 3 Done Done Sent false
20710 6250 Acked Flush 0 Done Done Sent false
20711 6251 Acked Read 5 Done Done Sent false
20712 6252 Acked Read 2 Done Done Sent false
20713 6253 Acked Write 7 Done Done Sent false
20714 6254 Acked Write 10 Done Done Sent false
20715 6255 Acked Read 4 Done Done Sent false
20716 6256 Acked Write 8 Done Done Sent false
20717 6257 Acked Read 7 Done Done Sent false
20718 6258 Acked Read 7 Done Done Sent false
20719 6259 Acked Read 2 Done Done Sent false
20720 6260 Acked Read 4 Done Done Sent false
20721 6261 Acked Read 7 Done Done Sent false
20722 6262 Acked Flush 0 Done Done Sent false
20723 6263 Acked Read 1 Done Done Sent false
20724 6264 Acked Read 1 Done Done Sent false
20725 6265 Acked Read 4 Done Done Sent false
20726 6266 Acked Write 3 Done Done Sent false
20727 6267 Acked Read 3 Done Done Sent false
20728 6268 Acked Write 5 Done Done Sent false
20729 6269 Acked Read 5 Done Done Sent false
20730 6270 Acked Read 7 Done Done Sent false
20731 6271 Acked Write 10 Done Done Sent false
20732 6272 Acked Flush 0 Done Done Sent false
20733 6273 Acked Write 8 Done Done Sent false
20734 6274 Acked Read 2 Done Done Sent false
20735 6275 Acked Read 2 Done Done Sent false
20736 6276 Acked Write 6 Done Done Sent false
20737 6277 Acked Read 5 Done Done Sent false
20738 6278 Acked Read 9 Done Done Sent false
20739 6279 Acked Read 8 Done Done Sent false
20740 6280 Acked Write 7 Done Done Sent false
20741 6281 Acked Read 4 Done Done Sent false
20742 6282 Acked Write 2 Done Done Sent false
20743 6283 Acked Flush 0 Done Done Sent false
20744 6284 Acked Read 2 Done Done Sent false
20745 6285 Acked Read 10 Done Done Sent false
20746 6286 Acked Flush 0 Done Done Sent false
20747 6287 Acked Read 1 Done Done Sent false
20748 6288 Acked Write 6 Done Done Sent false
20749 6289 Acked Read 9 Done Done Sent false
20750 6290 Acked Read 6 Done Done Sent false
20751 6291 Acked Flush 0 Done Done Sent false
20752 6292 Acked Read 4 Done Done Sent false
20753 6293 Acked Read 3 Done Done Sent false
20754 6294 Acked Write 3 Done Done Sent false
20755 6295 Acked Flush 0 Done Done Sent false
20756 6296 Acked Write 4 Done Done Sent false
20757 6297 Acked Write 10 Done Done Sent false
20758 6298 Acked Write 3 Done Done Sent false
20759 6299 Acked Write 7 Done Done Sent false
20760 6300 Acked Read 1 Done Done Sent false
20761 6301 Acked Read 9 Done Done Sent false
20762 6302 Acked Read 10 Done Done Sent false
20763 6303 Acked Write 1 Done Done Sent false
20764 6304 Acked Read 10 Done Done Sent false
20765 6305 Acked Read 1 Done Done Sent false
20766 6306 Acked Flush 0 Done Done Sent false
20767 6307 Acked Read 10 Done Done Sent false
20768 6308 Acked Write 2 Done Done Sent false
20769 6309 Acked Read 1 Done Done Sent false
20770 6310 Acked Read 3 Done Done Sent false
20771 6311 Acked Flush 0 Done Done Sent false
20772 6312 Acked Write 6 Done Done Sent false
20773 6313 Acked Read 7 Done Done Sent false
20774 6314 Acked Read 9 Done Done Sent false
20775 6315 Acked Read 4 Done Done Sent false
20776 6316 Acked Write 4 Done Done Sent false
20777 6317 Acked Write 3 Done Done Sent false
20778 6318 Acked Read 7 Done Done Sent false
20779 6319 Acked Read 10 Done Done Sent false
20780 6320 Acked Read 4 Done Done Sent false
20781 6321 Acked Read 1 Done Done Sent false
20782 6322 Acked Read 6 Done Done Sent false
20783 6323 Acked Write 3 Done Done Sent false
20784 6324 Acked Read 9 Done Done Sent false
20785 6325 Acked Write 2 Done Done Sent false
20786 6326 Acked Write 10 Done Done Sent false
20787 6327 Acked Write 9 Done Done Sent false
20788 6328 Acked Write 7 Done Done Sent false
20789 6329 Acked Flush 0 Done Done Sent false
20790 6330 Acked Write 9 Done Done Sent false
20791 6331 Acked Write 6 Done Done Sent false
20792 6332 Acked Write 9 Done Done Sent false
20793 6333 Acked Read 3 Done Done Sent false
20794 6334 Acked Read 4 Done Done Sent false
20795 6335 Acked Write 6 Done Done Sent false
20796 6336 Acked Read 7 Done Done Sent false
20797 6337 Acked Read 1 Done Done Sent false
20798 6338 Acked Read 1 Done Done Sent false
20799 6339 Acked Write 7 Done Done Sent false
20800 6340 Acked Write 4 Done Done Sent false
20801 6341 Acked Read 5 Done Done Sent false
20802 6342 Acked Read 1 Done Done Sent false
20803 6343 Acked Write 7 Done Done Sent false
20804 6344 Acked Read 4 Done Done Sent false
20805 6345 Acked Read 3 Done Done Sent false
20806 6346 Acked Flush 0 Done Done Sent false
20807 6347 Acked Write 7 Done Done Sent false
20808 6348 Acked Read 9 Done Done Sent false
20809 6349 Acked Read 2 Done Done Sent false
20810 6350 Acked Write 7 Done Done Sent false
20811 6351 Acked Flush 0 Done Done Sent false
20812 6352 Acked Flush 0 Done Done Sent false
20813 6353 Acked Read 2 Done Done Sent false
20814 6354 Acked Read 10 Done Done Sent false
20815 6355 Acked Write 3 Done Done Sent false
20816 6356 Acked Flush 0 Done Done Sent false
20817 6357 Acked Write 5 Done Done Sent false
20818 6358 Acked Flush 0 Done Done Sent false
20819 6359 Acked Write 7 Done Done Sent false
20820 6360 Acked Write 4 Done Done Sent false
20821 6361 Acked Write 8 Done Done Sent false
20822 6362 Acked Read 10 Done Done Sent false
20823 6363 Acked Read 5 Done Done Sent false
20824 6364 Acked Read 9 Done Done Sent false
20825 6365 Acked Read 2 Done Done Sent false
20826 6366 Acked Read 10 Done Done Sent false
20827 6367 Acked Write 8 Done Done Sent false
20828 6368 Acked Flush 0 Done Done Sent false
20829 6369 Acked Read 4 Done Done Sent false
20830 6370 Acked Write 2 Done Done Sent false
20831 6371 Acked Read 4 Done Done Sent false
20832 6372 Acked Read 6 Done Done Sent false
20833 6373 Acked Read 7 Done Done Sent false
20834 6374 Acked Read 5 Done Done Sent false
20835 6375 Acked Write 7 Done Done Sent false
20836 6376 Acked Write 4 Done Done Sent false
20837 6377 Acked Write 4 Done Done Sent false
20838 6378 Acked Flush 0 Done Done Sent false
20839 6379 Acked Write 1 Done Done Sent false
20840 6380 Acked Read 5 Done Done Sent false
20841 6381 Acked Write 5 Done Done Sent false
20842 6382 Acked Read 2 Done Done Sent false
20843 6383 Acked Write 7 Done Done Sent false
20844 6384 Acked Flush 0 Done Done Sent false
20845 6385 Acked Read 8 Done Done Sent false
20846 6386 Acked Read 1 Done Done Sent false
20847 6387 Acked Write 7 Done Done Sent false
20848 6388 Acked Write 1 Done Done Sent false
20849 6389 Acked Write 1 Done Done Sent false
20850 6390 Acked Read 8 Done Done Sent false
20851 6391 Acked Write 6 Done Done Sent false
20852 6392 Acked Read 8 Done Done Sent false
20853 6393 Acked Flush 0 Done Done Sent false
20854 6394 Acked Read 2 Done Done Sent false
20855 6395 Acked Write 10 Done Done Sent false
20856 6396 Acked Read 10 Done Done Sent false
20857 6397 Acked Write 4 Done Done Sent false
20858 6398 Acked Read 3 Done Done Sent false
20859 6399 Acked Flush 0 Done Done Sent false
20860 6400 Acked Read 10 Done Done Sent false
20861 6401 Acked Write 4 Done Done Sent false
20862 6402 Acked Read 7 Done Done Sent false
20863 6403 Acked Read 3 Done Done Sent false
20864 6404 Acked Write 7 Done Done Sent false
20865 6405 Acked Write 2 Done Done Sent false
20866 6406 Acked Read 1 Done Done Sent false
20867 6407 Acked Read 2 Done Done Sent false
20868 6408 Acked Flush 0 Done Done Sent false
20869 6409 Acked Read 6 Done Done Sent false
20870 6410 Acked Write 3 Done Done Sent false
20871 6411 Acked Flush 0 Done Done Sent false
20872 6412 Acked Read 5 Done Done Sent false
20873 6413 Acked Read 2 Done Done Sent false
20874 6414 Acked Write 5 Done Done Sent false
20875 6415 Acked Read 2 Done Done Sent false
20876 6416 Acked Read 5 Done Done Sent false
20877 6417 Acked Read 6 Done Done Sent false
20878 6418 Acked Write 9 Done Done Sent false
20879 6419 Acked Write 6 Done Done Sent false
20880 6420 Acked Flush 0 Done Done Sent false
20881 6421 Acked Write 10 Done Done Sent false
20882 6422 Acked Read 10 Done Done Sent false
20883 6423 Acked Read 2 Done Done Sent false
20884 6424 Acked Read 8 Done Done Sent false
20885 6425 Acked Write 5 Done Done Sent false
20886 6426 Acked Read 9 Done Done Sent false
20887 6427 Acked Read 9 Done Done Sent false
20888 6428 Acked Read 7 Done Done Sent false
20889 6429 Acked Write 5 Done Done Sent false
20890 6430 Acked Write 5 Done Done Sent false
20891 6431 Acked Read 4 Done Done Sent false
20892 6432 Acked Write 1 Done Done Sent false
20893 6433 Acked Write 6 Done Done Sent false
20894 6434 Acked Write 10 Done Done Sent false
20895 6435 Acked Write 4 Done Done Sent false
20896 6436 Acked Read 8 Done Done Sent false
20897 6437 Acked Read 6 Done Done Sent false
20898 6438 Acked Read 3 Done Done Sent false
20899 6439 Acked Write 8 Done Done Sent false
20900 6440 Acked Read 10 Done Done Sent false
20901 6441 Acked Flush 0 Done Done Sent false
20902 STATES DS:0 DS:1 DS:2 TOTAL
20903 Sent 0 0 302 302
20904 Done 302 302 0 604
20905 Skipped 0 0 0 0
20906 Error 0 0 0 0
20907 Last Flush: 6441 6441 6139
20908 Downstairs last five completed: 6139 6138 6137 6136 6135
20909 Upstairs last five completed: 6441 6440 6439 6438 6437
20910 CLIENT: Up:0 ds:302 act:2
209112024-11-07T14:07:40.844ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
209122024-11-07T14:07:40.844ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
209132024-11-07T14:07:40.844ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
209142024-11-07T14:07:40.844ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
209152024-11-07T14:07:40.844ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
20916 [8830] got start action
20917 [8830] I am going to restart
20918 Starting downstairs at port 8830
20919 Make output file at "/tmp/dsc/downstairs-8830.txt"
20920 [8830][2] reports Starting
20921 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1217)
20922 [8830][2] reports Running
20923 ----------------------------------------------------------------
20924 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20925 States: Active Active Offline
20926 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20927 6140 Acked Read 1 Done Done Sent false
20928 6141 Acked Read 7 Done Done Sent false
20929 6142 Acked Write 6 Done Done Sent false
20930 6143 Acked Write 5 Done Done Sent false
20931 6144 Acked Write 2 Done Done Sent false
20932 6145 Acked Read 8 Done Done Sent false
20933 6146 Acked Flush 0 Done Done Sent false
20934 6147 Acked Flush 0 Done Done Sent false
20935 6148 Acked Read 8 Done Done Sent false
20936 6149 Acked Flush 0 Done Done Sent false
20937 6150 Acked Write 2 Done Done Sent false
20938 6151 Acked Read 1 Done Done Sent false
20939 6152 Acked Write 4 Done Done Sent false
20940 6153 Acked Write 8 Done Done Sent false
20941 6154 Acked Read 5 Done Done Sent false
20942 6155 Acked Write 8 Done Done Sent false
20943 6156 Acked Read 4 Done Done Sent false
20944 6157 Acked Write 7 Done Done Sent false
20945 6158 Acked Flush 0 Done Done Sent false
20946 6159 Acked Read 9 Done Done Sent false
20947 6160 Acked Write 6 Done Done Sent false
20948 6161 Acked Flush 0 Done Done Sent false
20949 6162 Acked Write 6 Done Done Sent false
20950 6163 Acked Read 5 Done Done Sent false
20951 6164 Acked Flush 0 Done Done Sent false
20952 6165 Acked Flush 0 Done Done Sent false
20953 6166 Acked Read 4 Done Done Sent false
20954 6167 Acked Flush 0 Done Done Sent false
20955 6168 Acked Write 8 Done Done Sent false
20956 6169 Acked Read 4 Done Done Sent false
20957 6170 Acked Read 3 Done Done Sent false
20958 6171 Acked Write 8 Done Done Sent false
20959 6172 Acked Write 9 Done Done Sent false
20960 6173 Acked Write 1 Done Done Sent false
20961 6174 Acked Flush 0 Done Done Sent false
20962 6175 Acked Write 8 Done Done Sent false
20963 6176 Acked Flush 0 Done Done Sent false
20964 6177 Acked Read 4 Done Done Sent false
20965 6178 Acked Read 6 Done Done Sent false
20966 6179 Acked Read 9 Done Done Sent false
20967 6180 Acked Flush 0 Done Done Sent false
20968 6181 Acked Write 10 Done Done Sent false
20969 6182 Acked Write 5 Done Done Sent false
20970 6183 Acked Write 8 Done Done Sent false
20971 6184 Acked Read 4 Done Done Sent false
20972 6185 Acked Read 7 Done Done Sent false
20973 6186 Acked Read 3 Done Done Sent false
20974 6187 Acked Write 10 Done Done Sent false
20975 6188 Acked Read 4 Done Done Sent false
20976 6189 Acked Read 5 Done Done Sent false
20977 6190 Acked Write 10 Done Done Sent false
20978 6191 Acked Write 5 Done Done Sent false
20979 6192 Acked Read 8 Done Done Sent false
20980 6193 Acked Write 10 Done Done Sent false
20981 6194 Acked Read 7 Done Done Sent false
20982 6195 Acked Read 9 Done Done Sent false
20983 6196 Acked Read 8 Done Done Sent false
20984 6197 Acked Read 2 Done Done Sent false
20985 6198 Acked Write 8 Done Done Sent false
20986 6199 Acked Write 3 Done Done Sent false
20987 6200 Acked Read 7 Done Done Sent false
20988 6201 Acked Read 2 Done Done Sent false
20989 6202 Acked Write 9 Done Done Sent false
20990 6203 Acked Write 7 Done Done Sent false
20991 6204 Acked Flush 0 Done Done Sent false
20992 6205 Acked Write 2 Done Done Sent false
20993 6206 Acked Write 9 Done Done Sent false
20994 6207 Acked Read 4 Done Done Sent false
20995 6208 Acked Write 8 Done Done Sent false
20996 6209 Acked Write 2 Done Done Sent false
20997 6210 Acked Read 3 Done Done Sent false
20998 6211 Acked Read 1 Done Done Sent false
20999 6212 Acked Flush 0 Done Done Sent false
21000 6213 Acked Write 9 Done Done Sent false
21001 6214 Acked Write 3 Done Done Sent false
21002 6215 Acked Read 10 Done Done Sent false
21003 6216 Acked Flush 0 Done Done Sent false
21004 6217 Acked Write 10 Done Done Sent false
21005 6218 Acked Write 7 Done Done Sent false
21006 6219 Acked Read 2 Done Done Sent false
21007 6220 Acked Read 7 Done Done Sent false
21008 6221 Acked Read 6 Done Done Sent false
21009 6222 Acked Read 4 Done Done Sent false
21010 6223 Acked Write 1 Done Done Sent false
21011 6224 Acked Write 2 Done Done Sent false
21012 6225 Acked Write 1 Done Done Sent false
21013 6226 Acked Write 9 Done Done Sent false
21014 6227 Acked Read 6 Done Done Sent false
21015 6228 Acked Write 10 Done Done Sent false
21016 6229 Acked Read 7 Done Done Sent false
21017 6230 Acked Write 5 Done Done Sent false
21018 6231 Acked Read 9 Done Done Sent false
21019 6232 Acked Write 1 Done Done Sent false
21020 6233 Acked Read 2 Done Done Sent false
21021 6234 Acked Write 3 Done Done Sent false
21022 6235 Acked Read 10 Done Done Sent false
21023 6236 Acked Write 4 Done Done Sent false
21024 6237 Acked Read 2 Done Done Sent false
21025 6238 Acked Write 9 Done Done Sent false
21026 6239 Acked Read 3 Done Done Sent false
21027 6240 Acked Write 4 Done Done Sent false
21028 6241 Acked Read 6 Done Done Sent false
21029 6242 Acked Write 8 Done Done Sent false
21030 6243 Acked Write 1 Done Done Sent false
21031 6244 Acked Write 3 Done Done Sent false
21032 6245 Acked Read 1 Done Done Sent false
21033 6246 Acked Read 5 Done Done Sent false
21034 6247 Acked Read 4 Done Done Sent false
21035 6248 Acked Read 9 Done Done Sent false
21036 6249 Acked Write 3 Done Done Sent false
21037 6250 Acked Flush 0 Done Done Sent false
21038 6251 Acked Read 5 Done Done Sent false
21039 6252 Acked Read 2 Done Done Sent false
21040 6253 Acked Write 7 Done Done Sent false
21041 6254 Acked Write 10 Done Done Sent false
21042 6255 Acked Read 4 Done Done Sent false
21043 6256 Acked Write 8 Done Done Sent false
21044 6257 Acked Read 7 Done Done Sent false
21045 6258 Acked Read 7 Done Done Sent false
21046 6259 Acked Read 2 Done Done Sent false
21047 6260 Acked Read 4 Done Done Sent false
21048 6261 Acked Read 7 Done Done Sent false
21049 6262 Acked Flush 0 Done Done Sent false
21050 6263 Acked Read 1 Done Done Sent false
21051 6264 Acked Read 1 Done Done Sent false
21052 6265 Acked Read 4 Done Done Sent false
21053 6266 Acked Write 3 Done Done Sent false
21054 6267 Acked Read 3 Done Done Sent false
21055 6268 Acked Write 5 Done Done Sent false
21056 6269 Acked Read 5 Done Done Sent false
21057 6270 Acked Read 7 Done Done Sent false
21058 6271 Acked Write 10 Done Done Sent false
21059 6272 Acked Flush 0 Done Done Sent false
21060 6273 Acked Write 8 Done Done Sent false
21061 6274 Acked Read 2 Done Done Sent false
21062 6275 Acked Read 2 Done Done Sent false
21063 6276 Acked Write 6 Done Done Sent false
21064 6277 Acked Read 5 Done Done Sent false
21065 6278 Acked Read 9 Done Done Sent false
21066 6279 Acked Read 8 Done Done Sent false
21067 6280 Acked Write 7 Done Done Sent false
21068 6281 Acked Read 4 Done Done Sent false
21069 6282 Acked Write 2 Done Done Sent false
21070 6283 Acked Flush 0 Done Done Sent false
21071 6284 Acked Read 2 Done Done Sent false
21072 6285 Acked Read 10 Done Done Sent false
21073 6286 Acked Flush 0 Done Done Sent false
21074 6287 Acked Read 1 Done Done Sent false
21075 6288 Acked Write 6 Done Done Sent false
21076 6289 Acked Read 9 Done Done Sent false
21077 6290 Acked Read 6 Done Done Sent false
21078 6291 Acked Flush 0 Done Done Sent false
21079 6292 Acked Read 4 Done Done Sent false
21080 6293 Acked Read 3 Done Done Sent false
21081 6294 Acked Write 3 Done Done Sent false
21082 6295 Acked Flush 0 Done Done Sent false
21083 6296 Acked Write 4 Done Done Sent false
21084 6297 Acked Write 10 Done Done Sent false
21085 6298 Acked Write 3 Done Done Sent false
21086 6299 Acked Write 7 Done Done Sent false
21087 6300 Acked Read 1 Done Done Sent false
21088 6301 Acked Read 9 Done Done Sent false
21089 6302 Acked Read 10 Done Done Sent false
21090 6303 Acked Write 1 Done Done Sent false
21091 6304 Acked Read 10 Done Done Sent false
21092 6305 Acked Read 1 Done Done Sent false
21093 6306 Acked Flush 0 Done Done Sent false
21094 6307 Acked Read 10 Done Done Sent false
21095 6308 Acked Write 2 Done Done Sent false
21096 6309 Acked Read 1 Done Done Sent false
21097 6310 Acked Read 3 Done Done Sent false
21098 6311 Acked Flush 0 Done Done Sent false
21099 6312 Acked Write 6 Done Done Sent false
21100 6313 Acked Read 7 Done Done Sent false
21101 6314 Acked Read 9 Done Done Sent false
21102 6315 Acked Read 4 Done Done Sent false
21103 6316 Acked Write 4 Done Done Sent false
21104 6317 Acked Write 3 Done Done Sent false
21105 6318 Acked Read 7 Done Done Sent false
21106 6319 Acked Read 10 Done Done Sent false
21107 6320 Acked Read 4 Done Done Sent false
21108 6321 Acked Read 1 Done Done Sent false
21109 6322 Acked Read 6 Done Done Sent false
21110 6323 Acked Write 3 Done Done Sent false
21111 6324 Acked Read 9 Done Done Sent false
21112 6325 Acked Write 2 Done Done Sent false
21113 6326 Acked Write 10 Done Done Sent false
21114 6327 Acked Write 9 Done Done Sent false
21115 6328 Acked Write 7 Done Done Sent false
21116 6329 Acked Flush 0 Done Done Sent false
21117 6330 Acked Write 9 Done Done Sent false
21118 6331 Acked Write 6 Done Done Sent false
21119 6332 Acked Write 9 Done Done Sent false
21120 6333 Acked Read 3 Done Done Sent false
21121 6334 Acked Read 4 Done Done Sent false
21122 6335 Acked Write 6 Done Done Sent false
21123 6336 Acked Read 7 Done Done Sent false
21124 6337 Acked Read 1 Done Done Sent false
21125 6338 Acked Read 1 Done Done Sent false
21126 6339 Acked Write 7 Done Done Sent false
21127 6340 Acked Write 4 Done Done Sent false
21128 6341 Acked Read 5 Done Done Sent false
21129 6342 Acked Read 1 Done Done Sent false
21130 6343 Acked Write 7 Done Done Sent false
21131 6344 Acked Read 4 Done Done Sent false
21132 6345 Acked Read 3 Done Done Sent false
21133 6346 Acked Flush 0 Done Done Sent false
21134 6347 Acked Write 7 Done Done Sent false
21135 6348 Acked Read 9 Done Done Sent false
21136 6349 Acked Read 2 Done Done Sent false
21137 6350 Acked Write 7 Done Done Sent false
21138 6351 Acked Flush 0 Done Done Sent false
21139 6352 Acked Flush 0 Done Done Sent false
21140 6353 Acked Read 2 Done Done Sent false
21141 6354 Acked Read 10 Done Done Sent false
21142 6355 Acked Write 3 Done Done Sent false
21143 6356 Acked Flush 0 Done Done Sent false
21144 6357 Acked Write 5 Done Done Sent false
21145 6358 Acked Flush 0 Done Done Sent false
21146 6359 Acked Write 7 Done Done Sent false
21147 6360 Acked Write 4 Done Done Sent false
21148 6361 Acked Write 8 Done Done Sent false
21149 6362 Acked Read 10 Done Done Sent false
21150 6363 Acked Read 5 Done Done Sent false
21151 6364 Acked Read 9 Done Done Sent false
21152 6365 Acked Read 2 Done Done Sent false
21153 6366 Acked Read 10 Done Done Sent false
21154 6367 Acked Write 8 Done Done Sent false
21155 6368 Acked Flush 0 Done Done Sent false
21156 6369 Acked Read 4 Done Done Sent false
21157 6370 Acked Write 2 Done Done Sent false
21158 6371 Acked Read 4 Done Done Sent false
21159 6372 Acked Read 6 Done Done Sent false
21160 6373 Acked Read 7 Done Done Sent false
21161 6374 Acked Read 5 Done Done Sent false
21162 6375 Acked Write 7 Done Done Sent false
21163 6376 Acked Write 4 Done Done Sent false
21164 6377 Acked Write 4 Done Done Sent false
21165 6378 Acked Flush 0 Done Done Sent false
21166 6379 Acked Write 1 Done Done Sent false
21167 6380 Acked Read 5 Done Done Sent false
21168 6381 Acked Write 5 Done Done Sent false
21169 6382 Acked Read 2 Done Done Sent false
21170 6383 Acked Write 7 Done Done Sent false
21171 6384 Acked Flush 0 Done Done Sent false
21172 6385 Acked Read 8 Done Done Sent false
21173 6386 Acked Read 1 Done Done Sent false
21174 6387 Acked Write 7 Done Done Sent false
21175 6388 Acked Write 1 Done Done Sent false
21176 6389 Acked Write 1 Done Done Sent false
21177 6390 Acked Read 8 Done Done Sent false
21178 6391 Acked Write 6 Done Done Sent false
21179 6392 Acked Read 8 Done Done Sent false
21180 6393 Acked Flush 0 Done Done Sent false
21181 6394 Acked Read 2 Done Done Sent false
21182 6395 Acked Write 10 Done Done Sent false
21183 6396 Acked Read 10 Done Done Sent false
21184 6397 Acked Write 4 Done Done Sent false
21185 6398 Acked Read 3 Done Done Sent false
21186 6399 Acked Flush 0 Done Done Sent false
21187 6400 Acked Read 10 Done Done Sent false
21188 6401 Acked Write 4 Done Done Sent false
21189 6402 Acked Read 7 Done Done Sent false
21190 6403 Acked Read 3 Done Done Sent false
21191 6404 Acked Write 7 Done Done Sent false
21192 6405 Acked Write 2 Done Done Sent false
21193 6406 Acked Read 1 Done Done Sent false
21194 6407 Acked Read 2 Done Done Sent false
21195 6408 Acked Flush 0 Done Done Sent false
21196 6409 Acked Read 6 Done Done Sent false
21197 6410 Acked Write 3 Done Done Sent false
21198 6411 Acked Flush 0 Done Done Sent false
21199 6412 Acked Read 5 Done Done Sent false
21200 6413 Acked Read 2 Done Done Sent false
21201 6414 Acked Write 5 Done Done Sent false
21202 6415 Acked Read 2 Done Done Sent false
21203 6416 Acked Read 5 Done Done Sent false
21204 6417 Acked Read 6 Done Done Sent false
21205 6418 Acked Write 9 Done Done Sent false
21206 6419 Acked Write 6 Done Done Sent false
21207 6420 Acked Flush 0 Done Done Sent false
21208 6421 Acked Write 10 Done Done Sent false
21209 6422 Acked Read 10 Done Done Sent false
21210 6423 Acked Read 2 Done Done Sent false
21211 6424 Acked Read 8 Done Done Sent false
21212 6425 Acked Write 5 Done Done Sent false
21213 6426 Acked Read 9 Done Done Sent false
21214 6427 Acked Read 9 Done Done Sent false
21215 6428 Acked Read 7 Done Done Sent false
21216 6429 Acked Write 5 Done Done Sent false
21217 6430 Acked Write 5 Done Done Sent false
21218 6431 Acked Read 4 Done Done Sent false
21219 6432 Acked Write 1 Done Done Sent false
21220 6433 Acked Write 6 Done Done Sent false
21221 6434 Acked Write 10 Done Done Sent false
21222 6435 Acked Write 4 Done Done Sent false
21223 6436 Acked Read 8 Done Done Sent false
21224 6437 Acked Read 6 Done Done Sent false
21225 6438 Acked Read 3 Done Done Sent false
21226 6439 Acked Write 8 Done Done Sent false
21227 6440 Acked Read 10 Done Done Sent false
21228 6441 Acked Flush 0 Done Done Sent false
21229 STATES DS:0 DS:1 DS:2 TOTAL
21230 Sent 0 0 302 302
21231 Done 302 302 0 604
21232 Skipped 0 0 0 0
21233 Error 0 0 0 0
21234 Last Flush: 6441 6441 6139
21235 Downstairs last five completed: 6139 6138 6137 6136 6135
21236 Upstairs last five completed: 6441 6440 6439 6438 6437
21237 CLIENT: Up:0 ds:302 act:2
21238 ----------------------------------------------------------------
21239 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21240 States: Active Active Offline
21241 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21242 6140 Acked Read 1 Done Done Sent false
21243 6141 Acked Read 7 Done Done Sent false
21244 6142 Acked Write 6 Done Done Sent false
21245 6143 Acked Write 5 Done Done Sent false
21246 6144 Acked Write 2 Done Done Sent false
21247 6145 Acked Read 8 Done Done Sent false
21248 6146 Acked Flush 0 Done Done Sent false
21249 6147 Acked Flush 0 Done Done Sent false
21250 6148 Acked Read 8 Done Done Sent false
21251 6149 Acked Flush 0 Done Done Sent false
21252 6150 Acked Write 2 Done Done Sent false
21253 6151 Acked Read 1 Done Done Sent false
21254 6152 Acked Write 4 Done Done Sent false
21255 6153 Acked Write 8 Done Done Sent false
21256 6154 Acked Read 5 Done Done Sent false
21257 6155 Acked Write 8 Done Done Sent false
21258 6156 Acked Read 4 Done Done Sent false
21259 6157 Acked Write 7 Done Done Sent false
21260 6158 Acked Flush 0 Done Done Sent false
21261 6159 Acked Read 9 Done Done Sent false
21262 6160 Acked Write 6 Done Done Sent false
21263 6161 Acked Flush 0 Done Done Sent false
21264 6162 Acked Write 6 Done Done Sent false
21265 6163 Acked Read 5 Done Done Sent false
21266 6164 Acked Flush 0 Done Done Sent false
21267 6165 Acked Flush 0 Done Done Sent false
21268 6166 Acked Read 4 Done Done Sent false
21269 6167 Acked Flush 0 Done Done Sent false
21270 6168 Acked Write 8 Done Done Sent false
21271 6169 Acked Read 4 Done Done Sent false
21272 6170 Acked Read 3 Done Done Sent false
21273 6171 Acked Write 8 Done Done Sent false
21274 6172 Acked Write 9 Done Done Sent false
21275 6173 Acked Write 1 Done Done Sent false
21276 6174 Acked Flush 0 Done Done Sent false
21277 6175 Acked Write 8 Done Done Sent false
21278 6176 Acked Flush 0 Done Done Sent false
21279 6177 Acked Read 4 Done Done Sent false
21280 6178 Acked Read 6 Done Done Sent false
21281 6179 Acked Read 9 Done Done Sent false
21282 6180 Acked Flush 0 Done Done Sent false
21283 6181 Acked Write 10 Done Done Sent false
21284 6182 Acked Write 5 Done Done Sent false
21285 6183 Acked Write 8 Done Done Sent false
21286 6184 Acked Read 4 Done Done Sent false
21287 6185 Acked Read 7 Done Done Sent false
21288 6186 Acked Read 3 Done Done Sent false
21289 6187 Acked Write 10 Done Done Sent false
21290 6188 Acked Read 4 Done Done Sent false
21291 6189 Acked Read 5 Done Done Sent false
21292 6190 Acked Write 10 Done Done Sent false
21293 6191 Acked Write 5 Done Done Sent false
21294 6192 Acked Read 8 Done Done Sent false
21295 6193 Acked Write 10 Done Done Sent false
21296 6194 Acked Read 7 Done Done Sent false
21297 6195 Acked Read 9 Done Done Sent false
21298 6196 Acked Read 8 Done Done Sent false
21299 6197 Acked Read 2 Done Done Sent false
21300 6198 Acked Write 8 Done Done Sent false
21301 6199 Acked Write 3 Done Done Sent false
21302 6200 Acked Read 7 Done Done Sent false
21303 6201 Acked Read 2 Done Done Sent false
21304 6202 Acked Write 9 Done Done Sent false
21305 6203 Acked Write 7 Done Done Sent false
21306 6204 Acked Flush 0 Done Done Sent false
21307 6205 Acked Write 2 Done Done Sent false
21308 6206 Acked Write 9 Done Done Sent false
21309 6207 Acked Read 4 Done Done Sent false
21310 6208 Acked Write 8 Done Done Sent false
21311 6209 Acked Write 2 Done Done Sent false
21312 6210 Acked Read 3 Done Done Sent false
21313 6211 Acked Read 1 Done Done Sent false
21314 6212 Acked Flush 0 Done Done Sent false
21315 6213 Acked Write 9 Done Done Sent false
21316 6214 Acked Write 3 Done Done Sent false
21317 6215 Acked Read 10 Done Done Sent false
21318 6216 Acked Flush 0 Done Done Sent false
21319 6217 Acked Write 10 Done Done Sent false
21320 6218 Acked Write 7 Done Done Sent false
21321 6219 Acked Read 2 Done Done Sent false
21322 6220 Acked Read 7 Done Done Sent false
21323 6221 Acked Read 6 Done Done Sent false
21324 6222 Acked Read 4 Done Done Sent false
21325 6223 Acked Write 1 Done Done Sent false
21326 6224 Acked Write 2 Done Done Sent false
21327 6225 Acked Write 1 Done Done Sent false
21328 6226 Acked Write 9 Done Done Sent false
21329 6227 Acked Read 6 Done Done Sent false
21330 6228 Acked Write 10 Done Done Sent false
21331 6229 Acked Read 7 Done Done Sent false
21332 6230 Acked Write 5 Done Done Sent false
21333 6231 Acked Read 9 Done Done Sent false
21334 6232 Acked Write 1 Done Done Sent false
21335 6233 Acked Read 2 Done Done Sent false
21336 6234 Acked Write 3 Done Done Sent false
21337 6235 Acked Read 10 Done Done Sent false
21338 6236 Acked Write 4 Done Done Sent false
21339 6237 Acked Read 2 Done Done Sent false
21340 6238 Acked Write 9 Done Done Sent false
21341 6239 Acked Read 3 Done Done Sent false
21342 6240 Acked Write 4 Done Done Sent false
21343 6241 Acked Read 6 Done Done Sent false
21344 6242 Acked Write 8 Done Done Sent false
21345 6243 Acked Write 1 Done Done Sent false
21346 6244 Acked Write 3 Done Done Sent false
21347 6245 Acked Read 1 Done Done Sent false
21348 6246 Acked Read 5 Done Done Sent false
21349 6247 Acked Read 4 Done Done Sent false
21350 6248 Acked Read 9 Done Done Sent false
21351 6249 Acked Write 3 Done Done Sent false
21352 6250 Acked Flush 0 Done Done Sent false
21353 6251 Acked Read 5 Done Done Sent false
21354 6252 Acked Read 2 Done Done Sent false
21355 6253 Acked Write 7 Done Done Sent false
21356 6254 Acked Write 10 Done Done Sent false
21357 6255 Acked Read 4 Done Done Sent false
21358 6256 Acked Write 8 Done Done Sent false
21359 6257 Acked Read 7 Done Done Sent false
21360 6258 Acked Read 7 Done Done Sent false
21361 6259 Acked Read 2 Done Done Sent false
21362 6260 Acked Read 4 Done Done Sent false
21363 6261 Acked Read 7 Done Done Sent false
21364 6262 Acked Flush 0 Done Done Sent false
21365 6263 Acked Read 1 Done Done Sent false
21366 6264 Acked Read 1 Done Done Sent false
21367 6265 Acked Read 4 Done Done Sent false
21368 6266 Acked Write 3 Done Done Sent false
21369 6267 Acked Read 3 Done Done Sent false
21370 6268 Acked Write 5 Done Done Sent false
21371 6269 Acked Read 5 Done Done Sent false
21372 6270 Acked Read 7 Done Done Sent false
21373 6271 Acked Write 10 Done Done Sent false
21374 6272 Acked Flush 0 Done Done Sent false
21375 6273 Acked Write 8 Done Done Sent false
21376 6274 Acked Read 2 Done Done Sent false
21377 6275 Acked Read 2 Done Done Sent false
21378 6276 Acked Write 6 Done Done Sent false
21379 6277 Acked Read 5 Done Done Sent false
21380 6278 Acked Read 9 Done Done Sent false
21381 6279 Acked Read 8 Done Done Sent false
21382 6280 Acked Write 7 Done Done Sent false
21383 6281 Acked Read 4 Done Done Sent false
21384 6282 Acked Write 2 Done Done Sent false
21385 6283 Acked Flush 0 Done Done Sent false
21386 6284 Acked Read 2 Done Done Sent false
21387 6285 Acked Read 10 Done Done Sent false
21388 6286 Acked Flush 0 Done Done Sent false
21389 6287 Acked Read 1 Done Done Sent false
21390 6288 Acked Write 6 Done Done Sent false
21391 6289 Acked Read 9 Done Done Sent false
21392 6290 Acked Read 6 Done Done Sent false
21393 6291 Acked Flush 0 Done Done Sent false
21394 6292 Acked Read 4 Done Done Sent false
21395 6293 Acked Read 3 Done Done Sent false
21396 6294 Acked Write 3 Done Done Sent false
21397 6295 Acked Flush 0 Done Done Sent false
21398 6296 Acked Write 4 Done Done Sent false
21399 6297 Acked Write 10 Done Done Sent false
21400 6298 Acked Write 3 Done Done Sent false
21401 6299 Acked Write 7 Done Done Sent false
21402 6300 Acked Read 1 Done Done Sent false
21403 6301 Acked Read 9 Done Done Sent false
21404 6302 Acked Read 10 Done Done Sent false
21405 6303 Acked Write 1 Done Done Sent false
21406 6304 Acked Read 10 Done Done Sent false
21407 6305 Acked Read 1 Done Done Sent false
21408 6306 Acked Flush 0 Done Done Sent false
21409 6307 Acked Read 10 Done Done Sent false
21410 6308 Acked Write 2 Done Done Sent false
21411 6309 Acked Read 1 Done Done Sent false
21412 6310 Acked Read 3 Done Done Sent false
21413 6311 Acked Flush 0 Done Done Sent false
21414 6312 Acked Write 6 Done Done Sent false
21415 6313 Acked Read 7 Done Done Sent false
21416 6314 Acked Read 9 Done Done Sent false
21417 6315 Acked Read 4 Done Done Sent false
21418 6316 Acked Write 4 Done Done Sent false
21419 6317 Acked Write 3 Done Done Sent false
21420 6318 Acked Read 7 Done Done Sent false
21421 6319 Acked Read 10 Done Done Sent false
21422 6320 Acked Read 4 Done Done Sent false
21423 6321 Acked Read 1 Done Done Sent false
21424 6322 Acked Read 6 Done Done Sent false
21425 6323 Acked Write 3 Done Done Sent false
21426 6324 Acked Read 9 Done Done Sent false
21427 6325 Acked Write 2 Done Done Sent false
21428 6326 Acked Write 10 Done Done Sent false
21429 6327 Acked Write 9 Done Done Sent false
21430 6328 Acked Write 7 Done Done Sent false
21431 6329 Acked Flush 0 Done Done Sent false
21432 6330 Acked Write 9 Done Done Sent false
21433 6331 Acked Write 6 Done Done Sent false
21434 6332 Acked Write 9 Done Done Sent false
21435 6333 Acked Read 3 Done Done Sent false
21436 6334 Acked Read 4 Done Done Sent false
21437 6335 Acked Write 6 Done Done Sent false
21438 6336 Acked Read 7 Done Done Sent false
21439 6337 Acked Read 1 Done Done Sent false
21440 6338 Acked Read 1 Done Done Sent false
21441 6339 Acked Write 7 Done Done Sent false
21442 6340 Acked Write 4 Done Done Sent false
21443 6341 Acked Read 5 Done Done Sent false
21444 6342 Acked Read 1 Done Done Sent false
21445 6343 Acked Write 7 Done Done Sent false
21446 6344 Acked Read 4 Done Done Sent false
21447 6345 Acked Read 3 Done Done Sent false
21448 6346 Acked Flush 0 Done Done Sent false
21449 6347 Acked Write 7 Done Done Sent false
21450 6348 Acked Read 9 Done Done Sent false
21451 6349 Acked Read 2 Done Done Sent false
21452 6350 Acked Write 7 Done Done Sent false
21453 6351 Acked Flush 0 Done Done Sent false
21454 6352 Acked Flush 0 Done Done Sent false
21455 6353 Acked Read 2 Done Done Sent false
21456 6354 Acked Read 10 Done Done Sent false
21457 6355 Acked Write 3 Done Done Sent false
21458 6356 Acked Flush 0 Done Done Sent false
21459 6357 Acked Write 5 Done Done Sent false
21460 6358 Acked Flush 0 Done Done Sent false
21461 6359 Acked Write 7 Done Done Sent false
21462 6360 Acked Write 4 Done Done Sent false
21463 6361 Acked Write 8 Done Done Sent false
21464 6362 Acked Read 10 Done Done Sent false
21465 6363 Acked Read 5 Done Done Sent false
21466 6364 Acked Read 9 Done Done Sent false
21467 6365 Acked Read 2 Done Done Sent false
21468 6366 Acked Read 10 Done Done Sent false
21469 6367 Acked Write 8 Done Done Sent false
21470 6368 Acked Flush 0 Done Done Sent false
21471 6369 Acked Read 4 Done Done Sent false
21472 6370 Acked Write 2 Done Done Sent false
21473 6371 Acked Read 4 Done Done Sent false
21474 6372 Acked Read 6 Done Done Sent false
21475 6373 Acked Read 7 Done Done Sent false
21476 6374 Acked Read 5 Done Done Sent false
21477 6375 Acked Write 7 Done Done Sent false
21478 6376 Acked Write 4 Done Done Sent false
21479 6377 Acked Write 4 Done Done Sent false
21480 6378 Acked Flush 0 Done Done Sent false
21481 6379 Acked Write 1 Done Done Sent false
21482 6380 Acked Read 5 Done Done Sent false
21483 6381 Acked Write 5 Done Done Sent false
21484 6382 Acked Read 2 Done Done Sent false
21485 6383 Acked Write 7 Done Done Sent false
21486 6384 Acked Flush 0 Done Done Sent false
21487 6385 Acked Read 8 Done Done Sent false
21488 6386 Acked Read 1 Done Done Sent false
21489 6387 Acked Write 7 Done Done Sent false
21490 6388 Acked Write 1 Done Done Sent false
21491 6389 Acked Write 1 Done Done Sent false
21492 6390 Acked Read 8 Done Done Sent false
21493 6391 Acked Write 6 Done Done Sent false
21494 6392 Acked Read 8 Done Done Sent false
21495 6393 Acked Flush 0 Done Done Sent false
21496 6394 Acked Read 2 Done Done Sent false
21497 6395 Acked Write 10 Done Done Sent false
21498 6396 Acked Read 10 Done Done Sent false
21499 6397 Acked Write 4 Done Done Sent false
21500 6398 Acked Read 3 Done Done Sent false
21501 6399 Acked Flush 0 Done Done Sent false
21502 6400 Acked Read 10 Done Done Sent false
21503 6401 Acked Write 4 Done Done Sent false
21504 6402 Acked Read 7 Done Done Sent false
21505 6403 Acked Read 3 Done Done Sent false
21506 6404 Acked Write 7 Done Done Sent false
21507 6405 Acked Write 2 Done Done Sent false
21508 6406 Acked Read 1 Done Done Sent false
21509 6407 Acked Read 2 Done Done Sent false
21510 6408 Acked Flush 0 Done Done Sent false
21511 6409 Acked Read 6 Done Done Sent false
21512 6410 Acked Write 3 Done Done Sent false
21513 6411 Acked Flush 0 Done Done Sent false
21514 6412 Acked Read 5 Done Done Sent false
21515 6413 Acked Read 2 Done Done Sent false
21516 6414 Acked Write 5 Done Done Sent false
21517 6415 Acked Read 2 Done Done Sent false
21518 6416 Acked Read 5 Done Done Sent false
21519 6417 Acked Read 6 Done Done Sent false
21520 6418 Acked Write 9 Done Done Sent false
21521 6419 Acked Write 6 Done Done Sent false
21522 6420 Acked Flush 0 Done Done Sent false
21523 6421 Acked Write 10 Done Done Sent false
21524 6422 Acked Read 10 Done Done Sent false
21525 6423 Acked Read 2 Done Done Sent false
21526 6424 Acked Read 8 Done Done Sent false
21527 6425 Acked Write 5 Done Done Sent false
21528 6426 Acked Read 9 Done Done Sent false
21529 6427 Acked Read 9 Done Done Sent false
21530 6428 Acked Read 7 Done Done Sent false
21531 6429 Acked Write 5 Done Done Sent false
21532 6430 Acked Write 5 Done Done Sent false
21533 6431 Acked Read 4 Done Done Sent false
21534 6432 Acked Write 1 Done Done Sent false
21535 6433 Acked Write 6 Done Done Sent false
21536 6434 Acked Write 10 Done Done Sent false
21537 6435 Acked Write 4 Done Done Sent false
21538 6436 Acked Read 8 Done Done Sent false
21539 6437 Acked Read 6 Done Done Sent false
21540 6438 Acked Read 3 Done Done Sent false
21541 6439 Acked Write 8 Done Done Sent false
21542 6440 Acked Read 10 Done Done Sent false
21543 6441 Acked Flush 0 Done Done Sent false
21544 STATES DS:0 DS:1 DS:2 TOTAL
21545 Sent 0 0 302 302
21546 Done 302 302 0 604
21547 Skipped 0 0 0 0
21548 Error 0 0 0 0
21549 Last Flush: 6441 6441 6139
21550 Downstairs last five completed: 6139 6138 6137 6136 6135
21551 Upstairs last five completed: 6441 6440 6439 6438 6437
21552 CLIENT: Up:0 ds:302 act:2
215532024-11-07T14:07:50.845ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215542024-11-07T14:07:50.846ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50767) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215552024-11-07T14:07:50.847ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215562024-11-07T14:07:50.847ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215572024-11-07T14:07:50.847ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215582024-11-07T14:07:50.847ZINFOcrucible: send last flush ID to this DS: 6139 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215592024-11-07T14:07:50.847ZINFOcrucible: Replied this last flush ID: 6139 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215602024-11-07T14:07:50.847ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215612024-11-07T14:07:50.847ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215622024-11-07T14:07:50.847ZINFOcrucible: [2] client re-new 302 jobs since flush 6139 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215632024-11-07T14:07:50.848ZINFOcrucible: [2] Replayed 302 jobs since flush: 6139 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
21564 ----------------------------------------------------------------
21565 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21566 Downstairs last five completed: 6441 6440 6439 6438 6437
21567 Upstairs last five completed: 6441 6440 6439 6438 6437
21568 CLIENT: Up:0 ds:0 act:3
21569 Replay: All jobs finished
21570 Main task has work to do, go find it
21571 got dsc Stop(0)
21572 stop 0
21573 Nov 07 14:07:51.142[8810] Got stop action so:false kr:false
21574 INFO request completed, latency_us: 407, response_code: 204, uri: /stop/cid/0, method: POST, req_id: f724d57c-8382-4fea-a698-5af872ebf43e, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
215752024-11-07T14:07:51.143ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215762024-11-07T14:07:51.143ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215772024-11-07T14:07:51.143ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215782024-11-07T14:07:51.143ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215792024-11-07T14:07:51.143ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
215802024-11-07T14:07:51.143ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
21581 Nov 07 14:07:51.143 INFO request completed, latency_us: 174, response_code: 200, uri: /state/cid/0, method: GET, req_id: dcb8d268-9856-465d-a13e-b827e2abce0d, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
21582 [8810] Exited with: ExitStatus(unix_wait_status(9))
21583 [8810][0] reports Exit
21584 Nov 07 14:07:55.145 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/0, method: GET, req_id: 149d83ef-2364-4899-b0ec-9dee9c4bc276, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
21585 001/300 Read block 49 len 32768
21586 002/300 Write block 1254 len 4096 data: 4
21587 003/300 Write block 4192 len 12288 data: 3 3 3
21588 004/300 Read block 3612 len 36864
21589 005/300 Read block 3178 len 40960
21590 006/300 Write block 2461 len 8192 data: 2 2
21591 007/300 Flush
21592 008/300 Read block 380 len 4096
21593 009/300 Read block 3655 len 36864
21594 010/300 Read block 2774 len 16384
21595 011/300 Write block 2417 len 20480 data: 3 4 6 6 6
21596 012/300 Read block 1386 len 32768
21597 013/300 Read block 3623 len 16384
21598 014/300 Read block 3848 len 4096
21599 015/300 Read block 178 len 12288
21600 016/300 Flush
21601 017/300 Write block 1719 len 8192 data: 4 5
21602 018/300 Write block 4793 len 28672 data: 3 4 4 4 4 4 5
21603 019/300 Read block 3063 len 4096
21604 020/300 Read block 822 len 36864
21605 021/300 Write block 2143 len 16384 data: 4 6 4 5
21606 022/300 Read block 3477 len 8192
21607 023/300 Flush
21608 024/300 Read block 1805 len 20480
21609 025/300 Write block 4451 len 16384 data: 4 5 5 3
21610 026/300 Read block 303 len 16384
21611 027/300 Read block 2270 len 24576
21612 028/300 Read block 59 len 12288
21613 029/300 Write block 1111 len 24576 data: 2 2 2 3 3 4
21614 030/300 Read block 3383 len 24576
21615 031/300 Write block 557 len 36864 data: 4 3 3 3 3 3 4 4 3
21616 032/300 Write block 190 len 20480 data: 5 3 4 3 3
21617 033/300 Read block 4346 len 36864
21618 034/300 Write block 3021 len 24576 data: 4 5 5 5 5 4
21619 035/300 Write block 1376 len 32768 data: 6 5 3 3 3 3 3 3
21620 036/300 Read block 3343 len 28672
21621 037/300 Read block 4293 len 16384
21622 038/300 Read block 1819 len 32768
21623 039/300 Read block 3672 len 4096
21624 040/300 Read block 110 len 28672
21625 041/300 Read block 1310 len 12288
21626 042/300 Read block 4497 len 24576
21627 043/300 Read block 3052 len 36864
21628 044/300 Read block 3859 len 36864
21629 045/300 Read block 278 len 24576
21630 046/300 Write block 2269 len 20480 data: 5 6 6 6 6
21631 047/300 Read block 4789 len 40960
21632 048/300 Read block 375 len 36864
21633 049/300 Read block 83 len 8192
21634 050/300 Write block 3694 len 24576 data: 8 8 8 7 8 6
21635 051/300 Read block 1081 len 12288
21636 052/300 Read block 4501 len 20480
21637 053/300 Read block 520 len 8192
21638 054/300 Write block 1394 len 20480 data: 3 3 4 4 4
21639 055/300 Write block 4770 len 40960 data: 4 5 5 4 4 3 3 3 4 5
21640 056/300 Read block 4243 len 24576
21641 057/300 Write block 1287 len 20480 data: 3 3 4 4 3
21642 058/300 Read block 1096 len 16384
21643 059/300 Flush
21644 060/300 Read block 4212 len 40960
21645 061/300 Read block 193 len 8192
21646 062/300 Write block 1829 len 12288 data: 5 6 5
21647 063/300 Write block 1105 len 20480 data: 5 5 4 4 4
21648 064/300 Read block 3980 len 24576
21649 065/300 Flush
21650 066/300 Read block 4558 len 24576
21651 067/300 Write block 1385 len 20480 data: 5 5 5 5 3
21652 068/300 Read block 1692 len 28672
21653 069/300 Write block 2368 len 24576 data: 3 3 3 3 2 2
21654 070/300 Write block 3240 len 28672 data: 8 7 7 6 6 6 5
21655 071/300 Read block 1236 len 12288
21656 072/300 Read block 3826 len 32768
21657 073/300 Read block 3204 len 8192
21658 074/300 Read block 2613 len 4096
21659 075/300 Read block 4658 len 24576
21660 076/300 Read block 358 len 16384
21661 077/300 Write block 1328 len 32768 data: 6 7 8 7 8 9 7 7
21662 078/300 Read block 3304 len 20480
21663 079/300 Read block 3435 len 40960
21664 080/300 Write block 1711 len 16384 data: 6 5 5 5
21665 081/300 Write block 2401 len 4096 data: 3
21666 082/300 Write block 2984 len 36864 data: 4 4 4 4 4 4 4 3 3
21667 083/300 Read block 112 len 8192
21668 084/300 Read block 871 len 20480
21669 085/300 Read block 3401 len 8192
21670 086/300 Read block 2666 len 4096
21671 087/300 Read block 4639 len 12288
21672 088/300 Write block 532 len 32768 data: 6 6 5 4 3 3 4 4
21673 089/300 Read block 2971 len 32768
21674 090/300 Write block 3500 len 8192 data: 2 2
21675 091/300 Read block 4268 len 20480
21676 092/300 Write block 4964 len 8192 data: 4 5
21677 093/300 Flush
21678 094/300 Write block 3634 len 40960 data: 5 5 4 4 4 4 4 4 3 3
21679 095/300 Write block 2817 len 20480 data: 6 5 5 5 5
21680 096/300 Write block 3213 len 8192 data: 4 4
21681 097/300 Read block 1144 len 4096
21682 098/300 Read block 3436 len 12288
21683 099/300 Write block 2329 len 4096 data: 6
21684 100/300 Read block 1707 len 28672
21685 101/300 Read block 3006 len 36864
21686 102/300 Write block 3699 len 12288 data: 7 5 5
21687 103/300 Read block 4807 len 20480
21688 104/300 Write block 1882 len 16384 data: 4 5 6 5
21689 105/300 Write block 3716 len 32768 data: 7 8 9 9 9 8 8 6
21690 106/300 Read block 1016 len 16384
21691 107/300 Read block 1327 len 24576
21692 108/300 Write block 4230 len 40960 data: 6 5 5 5 6 6 5 6 7 8
21693 109/300 Write block 2534 len 12288 data: 4 4 5
21694 110/300 Write block 636 len 40960 data: 6 3 3 3 2 3 4 4 5 6
21695 111/300 Flush
21696 112/300 Write block 4562 len 16384 data: 4 3 3 3
21697 113/300 Write block 3544 len 20480 data: 4 5 5 4 5
21698 114/300 Write block 767 len 40960 data: 4 5 5 3 4 4 4 4 4 4
21699 115/300 Write block 3143 len 32768 data: 4 3 3 3 3 3 3 3
21700 116/300 Read block 8 len 36864
21701 117/300 Read block 2145 len 36864
21702 118/300 Read block 295 len 32768
21703 119/300 Read block 3022 len 40960
21704 120/300 Read block 387 len 24576
21705 121/300 Read block 4019 len 24576
21706 122/300 Flush
21707 123/300 Read block 3664 len 20480
21708 124/300 Read block 629 len 32768
21709 125/300 Write block 1481 len 36864 data: 7 5 5 5 5 4 4 4 4
21710 126/300 Write block 1979 len 8192 data: 4 4
21711 127/300 Read block 4818 len 28672
21712 128/300 Write block 1980 len 36864 data: 5 5 6 5 6 5 6 6 6
21713 129/300 Write block 3504 len 12288 data: 2 2 3
21714 130/300 Write block 3680 len 12288 data: 3 4 4
21715 131/300 Read block 1170 len 20480
21716 132/300 Read block 1239 len 12288
21717 133/300 Write block 1721 len 28672 data: 4 3 3 3 3 4 3
21718 134/300 Read block 775 len 40960
21719 135/300 Write block 2971 len 20480 data: 5 5 5 6 5
21720 136/300 Flush
21721 137/300 Flush
21722 138/300 Write block 3332 len 20480 data: 7 6 6 6 5
21723 139/300 Write block 566 len 4096 data: 4
21724 140/300 Read block 2674 len 36864
21725 141/300 Read block 4157 len 4096
21726 142/300 Write block 1757 len 36864 data: 4 3 3 2 2 3 3 3 4
21727 143/300 Flush
21728 144/300 Write block 3155 len 12288 data: 5 5 4
21729 145/300 Write block 4124 len 20480 data: 7 6 6 6 4
21730 146/300 Read block 1149 len 24576
21731 147/300 Read block 4603 len 24576
21732 148/300 Read block 926 len 32768
21733 149/300 Write block 899 len 4096 data: 10
21734 150/300 Read block 586 len 32768
21735 151/300 Write block 3543 len 40960 data: 5 5 6 6 5 6 5 4 5 7
21736 152/300 Read block 4220 len 24576
21737 153/300 Read block 521 len 32768
21738 154/300 Read block 3064 len 32768
21739 155/300 Read block 42 len 8192
21740 156/300 Read block 2704 len 24576
21741 157/300 Write block 3616 len 32768 data: 4 4 7 7 7 7 7 7
21742 158/300 Read block 2553 len 36864
21743 159/300 Read block 4745 len 8192
21744 160/300 Write block 3196 len 4096 data: 5
21745 161/300 Read block 4013 len 12288
21746 162/300 Write block 1762 len 8192 data: 4 4
21747 163/300 Write block 3801 len 20480 data: 6 5 6 6 4
21748 164/300 Read block 452 len 8192
21749 165/300 Flush
21750 166/300 Read block 909 len 20480
21751 167/300 Read block 853 len 8192
21752 168/300 Flush
21753 169/300 Read block 1624 len 12288
21754 170/300 Read block 790 len 12288
21755 171/300 Write block 4499 len 40960 data: 3 3 3 4 5 6 4 4 5 5
21756 172/300 Write block 4257 len 20480 data: 4 5 4 5 6
21757 173/300 Flush
21758 174/300 Read block 3620 len 28672
21759 175/300 Read block 4622 len 12288
21760 176/300 Read block 2529 len 4096
21761 177/300 Write block 159 len 24576 data: 6 7 7 6 6 5
21762 178/300 Write block 401 len 32768 data: 4 5 7 8 8 7 6 6
21763 179/300 Read block 1317 len 4096
21764 180/300 Read block 408 len 16384
21765 181/300 Write block 1782 len 12288 data: 3 3 3
21766 182/300 Read block 2848 len 32768
21767 183/300 Write block 2241 len 20480 data: 6 4 4 4 4
21768 184/300 Read block 2828 len 40960
21769 185/300 Read block 2446 len 12288
21770 186/300 Read block 1298 len 16384
21771 187/300 Read block 3767 len 20480
21772 188/300 Write block 860 len 8192 data: 5 5
21773 189/300 Read block 3193 len 8192
21774 190/300 Write block 1118 len 12288 data: 5 5 5
21775 191/300 Read block 2855 len 20480
21776 192/300 Read block 926 len 36864
21777 193/300 Read block 2223 len 40960
21778 194/300 Read block 1388 len 20480
21779 195/300 Read block 3515 len 28672
21780 196/300 Write block 2007 len 32768 data: 5 6 5 5 5 6 7 6
21781 197/300 Read block 3867 len 32768
21782 198/300 Write block 437 len 36864 data: 5 7 7 7 5 6 5 7 6
21783 199/300 Read block 424 len 20480
21784 200/300 Read block 1138 len 28672
21785 201/300 Read block 1703 len 24576
21786 202/300 Read block 282 len 24576
21787 203/300 Read block 1172 len 36864
21788 204/300 Read block 3025 len 32768
21789 205/300 Write block 2823 len 8192 data: 5 6
21790 206/300 Flush
21791 207/300 Write block 4237 len 24576 data: 7 8 9 7 6 5
21792 208/300 Write block 1054 len 12288 data: 4 4 3
21793 209/300 Read block 3470 len 20480
21794 210/300 Write block 980 len 4096 data: 6
21795 211/300 Read block 2074 len 28672
21796 212/300 Write block 4041 len 20480 data: 5 5 5 6 6
21797 213/300 Write block 4563 len 12288 data: 4 4 4
21798 214/300 Flush
21799 215/300 Write block 1673 len 8192 data: 5 4
21800 216/300 Read block 2120 len 40960
21801 217/300 Flush
21802 218/300 Read block 4127 len 12288
21803 219/300 Write block 4417 len 20480 data: 4 3 3 3 4
21804 220/300 Write block 2178 len 16384 data: 3 3 3 6
21805 221/300 Read block 4258 len 20480
21806 222/300 Write block 3977 len 20480 data: 4 3 3 2 3
21807 223/300 Read block 2709 len 28672
21808 224/300 Read block 3396 len 20480
21809 225/300 Write block 2203 len 40960 data: 3 3 4 5 5 4 4 3 3 3
21810 226/300 Read block 1689 len 16384
21811 227/300 Read block 3156 len 4096
21812 228/300 Flush
21813 229/300 Write block 1264 len 40960 data: 3 4 4 4 3 3 4 4 4 4
21814 230/300 Read block 4145 len 40960
21815 231/300 Read block 1562 len 36864
21816 232/300 Write block 168 len 28672 data: 4 4 3 3 4 4 4
21817 233/300 Flush
21818 234/300 Read block 2690 len 12288
21819 235/300 Write block 804 len 32768 data: 3 4 4 3 2 2 3 3
21820 236/300 Flush
21821 237/300 Flush
21822 238/300 Write block 3086 len 4096 data: 4
21823 239/300 Write block 1049 len 12288 data: 5 6 6
21824 240/300 Write block 3965 len 32768 data: 7 6 6 6 5 4 4 6
21825 241/300 Read block 2163 len 28672
21826 242/300 Read block 1285 len 8192
21827 243/300 Write block 1356 len 24576 data: 8 8 8 9 8 6
21828 244/300 Read block 898 len 36864
21829 245/300 Write block 1526 len 4096 data: 3
21830 246/300 Write block 79 len 28672 data: 4 4 4 5 5 5 5
21831 247/300 Read block 417 len 32768
21832 248/300 Flush
21833 249/300 Read block 4024 len 36864
21834 250/300 Read block 1022 len 32768
21835 251/300 Read block 891 len 36864
21836 252/300 Read block 1449 len 4096
21837 253/300 Write block 1840 len 20480 data: 3 4 4 6 5
21838 254/300 Flush
21839 255/300 Write block 946 len 8192 data: 3 3
21840 256/300 Read block 4296 len 36864
21841 257/300 Read block 807 len 28672
21842 258/300 Write block 1815 len 40960 data: 2 3 3 3 3 3 3 4 4 5
21843 259/300 Read block 4128 len 40960
21844 260/300 Read block 1338 len 28672
21845 261/300 Read block 877 len 4096
21846 262/300 Read block 3703 len 8192
21847 263/300 Read block 2554 len 20480
21848 264/300 Read block 2549 len 16384
21849 265/300 Read block 140 len 40960
21850 266/300 Flush
21851 267/300 Write block 989 len 32768 data: 4 2 2 3 3 4 4 3
21852 268/300 Read block 2043 len 40960
21853 269/300 Write block 4637 len 12288 data: 4 4 4
21854 270/300 Write block 2792 len 40960 data: 6 6 6 4 4 3 3 2 2 4
21855 271/300 Write block 1449 len 16384 data: 7 7 4 3
21856 272/300 Read block 1041 len 12288
21857 273/300 Write block 4819 len 16384 data: 4 4 3 3
21858 274/300 Read block 2543 len 36864
21859 275/300 Flush
21860 276/300 Read block 4440 len 8192
21861 277/300 Read block 3504 len 24576
21862 278/300 Read block 3281 len 8192
21863 279/300 Flush
21864 280/300 Read block 76 len 40960
21865 281/300 Read block 2370 len 8192
21866 282/300 Read block 3999 len 16384
21867 283/300 Read block 973 len 24576
21868 284/300 Read block 2571 len 20480
21869 285/300 Write block 1342 len 32768 data: 4 5 4 4 4 5 5 6
21870 286/300 Flush
21871 287/300 Write block 2949 len 4096 data: 5
21872 288/300 Read block 708 len 4096
21873 289/300 Write block 4728 len 16384 data: 6 7 7 8
21874 290/300 Write block 1611 len 16384 data: 3 2 4 4
21875 291/300 Read block 18 len 36864
21876 292/300 Write block 261 len 40960 data: 5 5 5 3 4 4 3 3 3 3
21877 293/300 Flush
21878 294/300 Read block 3240 len 20480
21879 295/300 Read block 3296 len 4096
21880 296/300 Read block 462 len 28672
21881 297/300 Write block 766 len 32768 data: 3 5 6 6 4 5 5 5
21882 298/300 Read block 3447 len 16384
21883 299/300 Write block 4073 len 12288 data: 2 2 3
21884 300/300 Write block 620 len 12288 data: 3 3 4
21885 301/300 Read block 4298 len 32768
21886 Main task has work to do, go find it
21887 got dsc Start(0)
21888 start 0
21889 Nov 07 14:07:55.440 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/0, method: POST, req_id: 643fc7ae-e8f5-4c26-b47e-ba6f1f996a7c, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
21890 Replay: started 0, returned:Ok(())
21891 ----------------------------------------------------------------
21892 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21893 States: Offline Active Active
21894 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21895 6442 Acked Read 8 Sent Done Done false
21896 6443 Acked Write 1 Sent Done Done false
21897 6444 Acked Write 3 Sent Done Done false
21898 6445 Acked Read 9 Sent Done Done false
21899 6446 Acked Read 10 Sent Done Done false
21900 6447 Acked Write 2 Sent Done Done false
21901 6448 Acked Flush 0 Sent Done Done false
21902 6449 Acked Read 1 Sent Done Done false
21903 6450 Acked Read 9 Sent Done Done false
21904 6451 Acked Read 4 Sent Done Done false
21905 6452 Acked Write 5 Sent Done Done false
21906 6453 Acked Read 8 Sent Done Done false
21907 6454 Acked Read 4 Sent Done Done false
21908 6455 Acked Read 1 Sent Done Done false
21909 6456 Acked Read 3 Sent Done Done false
21910 6457 Acked Flush 0 Sent Done Done false
21911 6458 Acked Write 2 Sent Done Done false
21912 6459 Acked Write 7 Sent Done Done false
21913 6460 Acked Read 1 Sent Done Done false
21914 6461 Acked Read 9 Sent Done Done false
21915 6462 Acked Write 4 Sent Done Done false
21916 6463 Acked Read 2 Sent Done Done false
21917 6464 Acked Flush 0 Sent Done Done false
21918 6465 Acked Read 5 Sent Done Done false
21919 6466 Acked Write 4 Sent Done Done false
21920 6467 Acked Read 4 Sent Done Done false
21921 6468 Acked Read 6 Sent Done Done false
21922 6469 Acked Read 3 Sent Done Done false
21923 6470 Acked Write 6 Sent Done Done false
21924 6471 Acked Read 6 Sent Done Done false
21925 6472 Acked Write 9 Sent Done Done false
21926 6473 Acked Write 5 Sent Done Done false
21927 6474 Acked Read 9 Sent Done Done false
21928 6475 Acked Write 6 Sent Done Done false
21929 6476 Acked Write 8 Sent Done Done false
21930 6477 Acked Read 7 Sent Done Done false
21931 6478 Acked Read 4 Sent Done Done false
21932 6479 Acked Read 8 Sent Done Done false
21933 6480 Acked Read 1 Sent Done Done false
21934 6481 Acked Read 7 Sent Done Done false
21935 6482 Acked Read 3 Sent Done Done false
21936 6483 Acked Read 6 Sent Done Done false
21937 6484 Acked Read 9 Sent Done Done false
21938 6485 Acked Read 9 Sent Done Done false
21939 6486 Acked Read 6 Sent Done Done false
21940 6487 Acked Write 5 Sent Done Done false
21941 6488 Acked Read 10 Sent Done Done false
21942 6489 Acked Read 9 Sent Done Done false
21943 6490 Acked Read 2 Sent Done Done false
21944 6491 Acked Write 6 Sent Done Done false
21945 6492 Acked Read 3 Sent Done Done false
21946 6493 Acked Read 5 Sent Done Done false
21947 6494 Acked Read 2 Sent Done Done false
21948 6495 Acked Write 5 Sent Done Done false
21949 6496 Acked Write 10 Sent Done Done false
21950 6497 Acked Read 6 Sent Done Done false
21951 6498 Acked Write 5 Sent Done Done false
21952 6499 Acked Read 4 Sent Done Done false
21953 6500 Acked Flush 0 Sent Done Done false
21954 6501 Acked Read 10 Sent Done Done false
21955 6502 Acked Read 2 Sent Done Done false
21956 6503 Acked Write 3 Sent Done Done false
21957 6504 Acked Write 5 Sent Done Done false
21958 6505 Acked Read 6 Sent Done Done false
21959 6506 Acked Flush 0 Sent Done Done false
21960 6507 Acked Read 6 Sent Done Done false
21961 6508 Acked Write 5 Sent Done Done false
21962 6509 Acked Read 7 Sent Done Done false
21963 6510 Acked Write 6 Sent Done Done false
21964 6511 Acked Write 7 Sent Done Done false
21965 6512 Acked Read 3 Sent Done Done false
21966 6513 Acked Read 8 Sent Done Done false
21967 6514 Acked Read 2 Sent Done Done false
21968 6515 Acked Read 1 Sent Done Done false
21969 6516 Acked Read 6 Sent Done Done false
21970 6517 Acked Read 4 Sent Done Done false
21971 6518 Acked Write 8 Sent Done Done false
21972 6519 Acked Read 5 Sent Done Done false
21973 6520 Acked Read 10 Sent Done Done false
21974 6521 Acked Write 4 Sent Done Done false
21975 6522 Acked Write 1 Sent Done Done false
21976 6523 Acked Write 9 Sent Done Done false
21977 6524 Acked Read 2 Sent Done Done false
21978 6525 Acked Read 5 Sent Done Done false
21979 6526 Acked Read 2 Sent Done Done false
21980 6527 Acked Read 1 Sent Done Done false
21981 6528 Acked Read 3 Sent Done Done false
21982 6529 Acked Write 8 Sent Done Done false
21983 6530 Acked Read 8 Sent Done Done false
21984 6531 Acked Write 2 Sent Done Done false
21985 6532 Acked Read 5 Sent Done Done false
21986 6533 Acked Write 2 Sent Done Done false
21987 6534 Acked Flush 0 Sent Done Done false
21988 6535 Acked Write 10 Sent Done Done false
21989 6536 Acked Write 5 Sent Done Done false
21990 6537 Acked Write 2 Sent Done Done false
21991 6538 Acked Read 1 Sent Done Done false
21992 6539 Acked Read 3 Sent Done Done false
21993 6540 Acked Write 1 Sent Done Done false
21994 6541 Acked Read 7 Sent Done Done false
21995 6542 Acked Read 9 Sent Done Done false
21996 6543 Acked Write 3 Sent Done Done false
21997 6544 Acked Read 5 Sent Done Done false
21998 6545 Acked Write 4 Sent Done Done false
21999 6546 Acked Write 8 Sent Done Done false
22000 6547 Acked Read 4 Sent Done Done false
22001 6548 Acked Read 6 Sent Done Done false
22002 6549 Acked Write 10 Sent Done Done false
22003 6550 Acked Write 3 Sent Done Done false
22004 6551 Acked Write 10 Sent Done Done false
22005 6552 Acked Flush 0 Sent Done Done false
22006 6553 Acked Write 4 Sent Done Done false
22007 6554 Acked Write 5 Sent Done Done false
22008 6555 Acked Write 10 Sent Done Done false
22009 6556 Acked Write 8 Sent Done Done false
22010 6557 Acked Read 9 Sent Done Done false
22011 6558 Acked Read 9 Sent Done Done false
22012 6559 Acked Read 8 Sent Done Done false
22013 6560 Acked Read 10 Sent Done Done false
22014 6561 Acked Read 6 Sent Done Done false
22015 6562 Acked Read 6 Sent Done Done false
22016 6563 Acked Flush 0 Sent Done Done false
22017 6564 Acked Read 5 Sent Done Done false
22018 6565 Acked Read 8 Sent Done Done false
22019 6566 Acked Write 9 Sent Done Done false
22020 6567 Acked Write 2 Sent Done Done false
22021 6568 Acked Read 7 Sent Done Done false
22022 6569 Acked Write 9 Sent Done Done false
22023 6570 Acked Write 3 Sent Done Done false
22024 6571 Acked Write 3 Sent Done Done false
22025 6572 Acked Read 5 Sent Done Done false
22026 6573 Acked Read 3 Sent Done Done false
22027 6574 Acked Write 7 Sent Done Done false
22028 6575 Acked Read 10 Sent Done Done false
22029 6576 Acked Write 5 Sent Done Done false
22030 6577 Acked Flush 0 Sent Done Done false
22031 6578 Acked Flush 0 Sent Done Done false
22032 6579 Acked Write 5 Sent Done Done false
22033 6580 Acked Write 1 Sent Done Done false
22034 6581 Acked Read 9 Sent Done Done false
22035 6582 Acked Read 1 Sent Done Done false
22036 6583 Acked Write 9 Sent Done Done false
22037 6584 Acked Flush 0 Sent Done Done false
22038 6585 Acked Write 3 Sent Done Done false
22039 6586 Acked Write 5 Sent Done Done false
22040 6587 Acked Read 6 Sent Done Done false
22041 6588 Acked Read 6 Sent Done Done false
22042 6589 Acked Read 8 Sent Done Done false
22043 6590 Acked Write 1 Sent Done Done false
22044 6591 Acked Read 8 Sent Done Done false
22045 6592 Acked Write 10 Sent Done Done false
22046 6593 Acked Read 6 Sent Done Done false
22047 6594 Acked Read 8 Sent Done Done false
22048 6595 Acked Read 8 Sent Done Done false
22049 6596 Acked Read 2 Sent Done Done false
22050 6597 Acked Read 6 Sent Done Done false
22051 6598 Acked Write 8 Sent Done Done false
22052 6599 Acked Read 9 Sent Done Done false
22053 6600 Acked Flush 0 Sent Done Done false
22054 6601 Acked Read 2 Sent Done Done false
22055 6602 Acked Write 1 Sent Done Done false
22056 6603 Acked Read 3 Sent Done Done false
22057 6604 Acked Write 2 Sent Done Done false
22058 6605 Acked Write 5 Sent Done Done false
22059 6606 Acked Read 2 Sent Done Done false
22060 6607 Acked Flush 0 Sent Done Done false
22061 6608 Acked Read 5 Sent Done Done false
22062 6609 Acked Read 2 Sent Done Done false
22063 6610 Acked Flush 0 Sent Done Done false
22064 6611 Acked Read 3 Sent Done Done false
22065 6612 Acked Read 3 Sent Done Done false
22066 6613 Acked Write 10 Sent Done Done false
22067 6614 Acked Write 5 Sent Done Done false
22068 6615 Acked Flush 0 Sent Done Done false
22069 6616 Acked Read 7 Sent Done Done false
22070 6617 Acked Read 3 Sent Done Done false
22071 6618 Acked Read 1 Sent Done Done false
22072 6619 Acked Write 6 Sent Done Done false
22073 6620 Acked Write 8 Sent Done Done false
22074 6621 Acked Read 1 Sent Done Done false
22075 6622 Acked Read 4 Sent Done Done false
22076 6623 Acked Write 3 Sent Done Done false
22077 6624 Acked Read 8 Sent Done Done false
22078 6625 Acked Write 5 Sent Done Done false
22079 6626 Acked Read 10 Sent Done Done false
22080 6627 Acked Read 3 Sent Done Done false
22081 6628 Acked Read 4 Sent Done Done false
22082 6629 Acked Read 5 Sent Done Done false
22083 6630 Acked Write 2 Sent Done Done false
22084 6631 Acked Read 2 Sent Done Done false
22085 6632 Acked Write 3 Sent Done Done false
22086 6633 Acked Read 5 Sent Done Done false
22087 6634 Acked Read 9 Sent Done Done false
22088 6635 Acked Read 10 Sent Done Done false
22089 6636 Acked Read 5 Sent Done Done false
22090 6637 Acked Read 7 Sent Done Done false
22091 6638 Acked Write 8 Sent Done Done false
22092 6639 Acked Read 8 Sent Done Done false
22093 6640 Acked Write 9 Sent Done Done false
22094 6641 Acked Read 5 Sent Done Done false
22095 6642 Acked Read 7 Sent Done Done false
22096 6643 Acked Read 6 Sent Done Done false
22097 6644 Acked Read 6 Sent Done Done false
22098 6645 Acked Read 9 Sent Done Done false
22099 6646 Acked Read 8 Sent Done Done false
22100 6647 Acked Write 2 Sent Done Done false
22101 6648 Acked Flush 0 Sent Done Done false
22102 6649 Acked Write 6 Sent Done Done false
22103 6650 Acked Write 3 Sent Done Done false
22104 6651 Acked Read 5 Sent Done Done false
22105 6652 Acked Write 1 Sent Done Done false
22106 6653 Acked Read 7 Sent Done Done false
22107 6654 Acked Write 5 Sent Done Done false
22108 6655 Acked Write 3 Sent Done Done false
22109 6656 Acked Flush 0 Sent Done Done false
22110 6657 Acked Write 2 Sent Done Done false
22111 6658 Acked Read 10 Sent Done Done false
22112 6659 Acked Flush 0 Sent Done Done false
22113 6660 Acked Read 3 Sent Done Done false
22114 6661 Acked Write 5 Sent Done Done false
22115 6662 Acked Write 4 Sent Done Done false
22116 6663 Acked Read 5 Sent Done Done false
22117 6664 Acked Write 5 Sent Done Done false
22118 6665 Acked Read 7 Sent Done Done false
22119 6666 Acked Read 5 Sent Done Done false
22120 6667 Acked Write 10 Sent Done Done false
22121 6668 Acked Read 4 Sent Done Done false
22122 6669 Acked Read 1 Sent Done Done false
22123 6670 Acked Flush 0 Sent Done Done false
22124 6671 Acked Write 10 Sent Done Done false
22125 6672 Acked Read 10 Sent Done Done false
22126 6673 Acked Read 9 Sent Done Done false
22127 6674 Acked Write 7 Sent Done Done false
22128 6675 Acked Flush 0 Sent Done Done false
22129 6676 Acked Read 3 Sent Done Done false
22130 6677 Acked Write 8 Sent Done Done false
22131 6678 Acked Flush 0 Sent Done Done false
22132 6679 Acked Flush 0 Sent Done Done false
22133 6680 Acked Write 1 Sent Done Done false
22134 6681 Acked Write 3 Sent Done Done false
22135 6682 Acked Write 8 Sent Done Done false
22136 6683 Acked Read 7 Sent Done Done false
22137 6684 Acked Read 2 Sent Done Done false
22138 6685 Acked Write 6 Sent Done Done false
22139 6686 Acked Read 9 Sent Done Done false
22140 6687 Acked Write 1 Sent Done Done false
22141 6688 Acked Write 7 Sent Done Done false
22142 6689 Acked Read 8 Sent Done Done false
22143 6690 Acked Flush 0 Sent Done Done false
22144 6691 Acked Read 9 Sent Done Done false
22145 6692 Acked Read 8 Sent Done Done false
22146 6693 Acked Read 9 Sent Done Done false
22147 6694 Acked Read 1 Sent Done Done false
22148 6695 Acked Write 5 Sent Done Done false
22149 6696 Acked Flush 0 Sent Done Done false
22150 6697 Acked Write 2 Sent Done Done false
22151 6698 Acked Read 9 Sent Done Done false
22152 6699 Acked Read 7 Sent Done Done false
22153 6700 Acked Write 10 Sent Done Done false
22154 6701 Acked Read 10 Sent Done Done false
22155 6702 Acked Read 7 Sent Done Done false
22156 6703 Acked Read 1 Sent Done Done false
22157 6704 Acked Read 2 Sent Done Done false
22158 6705 Acked Read 5 Sent Done Done false
22159 6706 Acked Read 4 Sent Done Done false
22160 6707 Acked Read 10 Sent Done Done false
22161 6708 Acked Flush 0 Sent Done Done false
22162 6709 Acked Write 8 Sent Done Done false
22163 6710 Acked Read 10 Sent Done Done false
22164 6711 Acked Write 3 Sent Done Done false
22165 6712 Acked Write 10 Sent Done Done false
22166 6713 Acked Write 4 Sent Done Done false
22167 6714 Acked Read 3 Sent Done Done false
22168 6715 Acked Write 4 Sent Done Done false
22169 6716 Acked Read 9 Sent Done Done false
22170 6717 Acked Flush 0 Sent Done Done false
22171 6718 Acked Read 2 Sent Done Done false
22172 6719 Acked Read 6 Sent Done Done false
22173 6720 Acked Read 2 Sent Done Done false
22174 6721 Acked Flush 0 Sent Done Done false
22175 6722 Acked Read 10 Sent Done Done false
22176 6723 Acked Read 2 Sent Done Done false
22177 6724 Acked Read 4 Sent Done Done false
22178 6725 Acked Read 6 Sent Done Done false
22179 6726 Acked Read 5 Sent Done Done false
22180 6727 Acked Write 8 Sent Done Done false
22181 6728 Acked Flush 0 Sent Done Done false
22182 6729 Acked Write 1 Sent Done Done false
22183 6730 Acked Read 1 Sent Done Done false
22184 6731 Acked Write 4 Sent Done Done false
22185 6732 Acked Write 4 Sent Done Done false
22186 6733 Acked Read 9 Sent Done Done false
22187 6734 Acked Write 10 Sent Done Done false
22188 6735 Acked Flush 0 Sent Done Done false
22189 6736 Acked Read 5 Sent Done Done false
22190 6737 Acked Read 1 Sent Done Done false
22191 6738 Acked Read 7 Sent Done Done false
22192 6739 Acked Write 8 Sent Done Done false
22193 6740 Acked Read 4 Sent Done Done false
22194 6741 Acked Write 3 Sent Done Done false
22195 6742 Acked Write 3 Sent Done Done false
22196 6743 Acked Read 8 Sent Done Done false
22197 STATES DS:0 DS:1 DS:2 TOTAL
22198 Sent 302 0 0 302
22199 Done 0 302 302 604
22200 Skipped 0 0 0 0
22201 Error 0 0 0 0
22202 Last Flush: 6441 6735 6735
22203 Downstairs last five completed: 6441 6440 6439 6438 6437
22204 Upstairs last five completed: 6743 6742 6741 6740 6739
22205 CLIENT: Up:0 ds:302 act:2
22206 [8810] got start action
22207 [8810] I am going to restart
22208 Starting downstairs at port 8810
22209 Make output file at "/tmp/dsc/downstairs-8810.txt"
22210 [8810][0] reports Starting
22211 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1218)
22212 [8810][0] reports Running
22213 ----------------------------------------------------------------
22214 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
22215 States: Offline Active Active
22216 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22217 6442 Acked Read 8 Sent Done Done false
22218 6443 Acked Write 1 Sent Done Done false
22219 6444 Acked Write 3 Sent Done Done false
22220 6445 Acked Read 9 Sent Done Done false
22221 6446 Acked Read 10 Sent Done Done false
22222 6447 Acked Write 2 Sent Done Done false
22223 6448 Acked Flush 0 Sent Done Done false
22224 6449 Acked Read 1 Sent Done Done false
22225 6450 Acked Read 9 Sent Done Done false
22226 6451 Acked Read 4 Sent Done Done false
22227 6452 Acked Write 5 Sent Done Done false
22228 6453 Acked Read 8 Sent Done Done false
22229 6454 Acked Read 4 Sent Done Done false
22230 6455 Acked Read 1 Sent Done Done false
22231 6456 Acked Read 3 Sent Done Done false
22232 6457 Acked Flush 0 Sent Done Done false
22233 6458 Acked Write 2 Sent Done Done false
22234 6459 Acked Write 7 Sent Done Done false
22235 6460 Acked Read 1 Sent Done Done false
22236 6461 Acked Read 9 Sent Done Done false
22237 6462 Acked Write 4 Sent Done Done false
22238 6463 Acked Read 2 Sent Done Done false
22239 6464 Acked Flush 0 Sent Done Done false
22240 6465 Acked Read 5 Sent Done Done false
22241 6466 Acked Write 4 Sent Done Done false
22242 6467 Acked Read 4 Sent Done Done false
22243 6468 Acked Read 6 Sent Done Done false
22244 6469 Acked Read 3 Sent Done Done false
22245 6470 Acked Write 6 Sent Done Done false
22246 6471 Acked Read 6 Sent Done Done false
22247 6472 Acked Write 9 Sent Done Done false
22248 6473 Acked Write 5 Sent Done Done false
22249 6474 Acked Read 9 Sent Done Done false
22250 6475 Acked Write 6 Sent Done Done false
22251 6476 Acked Write 8 Sent Done Done false
22252 6477 Acked Read 7 Sent Done Done false
22253 6478 Acked Read 4 Sent Done Done false
22254 6479 Acked Read 8 Sent Done Done false
22255 6480 Acked Read 1 Sent Done Done false
22256 6481 Acked Read 7 Sent Done Done false
22257 6482 Acked Read 3 Sent Done Done false
22258 6483 Acked Read 6 Sent Done Done false
22259 6484 Acked Read 9 Sent Done Done false
22260 6485 Acked Read 9 Sent Done Done false
22261 6486 Acked Read 6 Sent Done Done false
22262 6487 Acked Write 5 Sent Done Done false
22263 6488 Acked Read 10 Sent Done Done false
22264 6489 Acked Read 9 Sent Done Done false
22265 6490 Acked Read 2 Sent Done Done false
22266 6491 Acked Write 6 Sent Done Done false
22267 6492 Acked Read 3 Sent Done Done false
22268 6493 Acked Read 5 Sent Done Done false
22269 6494 Acked Read 2 Sent Done Done false
22270 6495 Acked Write 5 Sent Done Done false
22271 6496 Acked Write 10 Sent Done Done false
22272 6497 Acked Read 6 Sent Done Done false
22273 6498 Acked Write 5 Sent Done Done false
22274 6499 Acked Read 4 Sent Done Done false
22275 6500 Acked Flush 0 Sent Done Done false
22276 6501 Acked Read 10 Sent Done Done false
22277 6502 Acked Read 2 Sent Done Done false
22278 6503 Acked Write 3 Sent Done Done false
22279 6504 Acked Write 5 Sent Done Done false
22280 6505 Acked Read 6 Sent Done Done false
22281 6506 Acked Flush 0 Sent Done Done false
22282 6507 Acked Read 6 Sent Done Done false
22283 6508 Acked Write 5 Sent Done Done false
22284 6509 Acked Read 7 Sent Done Done false
22285 6510 Acked Write 6 Sent Done Done false
22286 6511 Acked Write 7 Sent Done Done false
22287 6512 Acked Read 3 Sent Done Done false
22288 6513 Acked Read 8 Sent Done Done false
22289 6514 Acked Read 2 Sent Done Done false
22290 6515 Acked Read 1 Sent Done Done false
22291 6516 Acked Read 6 Sent Done Done false
22292 6517 Acked Read 4 Sent Done Done false
22293 6518 Acked Write 8 Sent Done Done false
22294 6519 Acked Read 5 Sent Done Done false
22295 6520 Acked Read 10 Sent Done Done false
22296 6521 Acked Write 4 Sent Done Done false
22297 6522 Acked Write 1 Sent Done Done false
22298 6523 Acked Write 9 Sent Done Done false
22299 6524 Acked Read 2 Sent Done Done false
22300 6525 Acked Read 5 Sent Done Done false
22301 6526 Acked Read 2 Sent Done Done false
22302 6527 Acked Read 1 Sent Done Done false
22303 6528 Acked Read 3 Sent Done Done false
22304 6529 Acked Write 8 Sent Done Done false
22305 6530 Acked Read 8 Sent Done Done false
22306 6531 Acked Write 2 Sent Done Done false
22307 6532 Acked Read 5 Sent Done Done false
22308 6533 Acked Write 2 Sent Done Done false
22309 6534 Acked Flush 0 Sent Done Done false
22310 6535 Acked Write 10 Sent Done Done false
22311 6536 Acked Write 5 Sent Done Done false
22312 6537 Acked Write 2 Sent Done Done false
22313 6538 Acked Read 1 Sent Done Done false
22314 6539 Acked Read 3 Sent Done Done false
22315 6540 Acked Write 1 Sent Done Done false
22316 6541 Acked Read 7 Sent Done Done false
22317 6542 Acked Read 9 Sent Done Done false
22318 6543 Acked Write 3 Sent Done Done false
22319 6544 Acked Read 5 Sent Done Done false
22320 6545 Acked Write 4 Sent Done Done false
22321 6546 Acked Write 8 Sent Done Done false
22322 6547 Acked Read 4 Sent Done Done false
22323 6548 Acked Read 6 Sent Done Done false
22324 6549 Acked Write 10 Sent Done Done false
22325 6550 Acked Write 3 Sent Done Done false
22326 6551 Acked Write 10 Sent Done Done false
22327 6552 Acked Flush 0 Sent Done Done false
22328 6553 Acked Write 4 Sent Done Done false
22329 6554 Acked Write 5 Sent Done Done false
22330 6555 Acked Write 10 Sent Done Done false
22331 6556 Acked Write 8 Sent Done Done false
22332 6557 Acked Read 9 Sent Done Done false
22333 6558 Acked Read 9 Sent Done Done false
22334 6559 Acked Read 8 Sent Done Done false
22335 6560 Acked Read 10 Sent Done Done false
22336 6561 Acked Read 6 Sent Done Done false
22337 6562 Acked Read 6 Sent Done Done false
22338 6563 Acked Flush 0 Sent Done Done false
22339 6564 Acked Read 5 Sent Done Done false
22340 6565 Acked Read 8 Sent Done Done false
22341 6566 Acked Write 9 Sent Done Done false
22342 6567 Acked Write 2 Sent Done Done false
22343 6568 Acked Read 7 Sent Done Done false
22344 6569 Acked Write 9 Sent Done Done false
22345 6570 Acked Write 3 Sent Done Done false
22346 6571 Acked Write 3 Sent Done Done false
22347 6572 Acked Read 5 Sent Done Done false
22348 6573 Acked Read 3 Sent Done Done false
22349 6574 Acked Write 7 Sent Done Done false
22350 6575 Acked Read 10 Sent Done Done false
22351 6576 Acked Write 5 Sent Done Done false
22352 6577 Acked Flush 0 Sent Done Done false
22353 6578 Acked Flush 0 Sent Done Done false
22354 6579 Acked Write 5 Sent Done Done false
22355 6580 Acked Write 1 Sent Done Done false
22356 6581 Acked Read 9 Sent Done Done false
22357 6582 Acked Read 1 Sent Done Done false
22358 6583 Acked Write 9 Sent Done Done false
22359 6584 Acked Flush 0 Sent Done Done false
22360 6585 Acked Write 3 Sent Done Done false
22361 6586 Acked Write 5 Sent Done Done false
22362 6587 Acked Read 6 Sent Done Done false
22363 6588 Acked Read 6 Sent Done Done false
22364 6589 Acked Read 8 Sent Done Done false
22365 6590 Acked Write 1 Sent Done Done false
22366 6591 Acked Read 8 Sent Done Done false
22367 6592 Acked Write 10 Sent Done Done false
22368 6593 Acked Read 6 Sent Done Done false
22369 6594 Acked Read 8 Sent Done Done false
22370 6595 Acked Read 8 Sent Done Done false
22371 6596 Acked Read 2 Sent Done Done false
22372 6597 Acked Read 6 Sent Done Done false
22373 6598 Acked Write 8 Sent Done Done false
22374 6599 Acked Read 9 Sent Done Done false
22375 6600 Acked Flush 0 Sent Done Done false
22376 6601 Acked Read 2 Sent Done Done false
22377 6602 Acked Write 1 Sent Done Done false
22378 6603 Acked Read 3 Sent Done Done false
22379 6604 Acked Write 2 Sent Done Done false
22380 6605 Acked Write 5 Sent Done Done false
22381 6606 Acked Read 2 Sent Done Done false
22382 6607 Acked Flush 0 Sent Done Done false
22383 6608 Acked Read 5 Sent Done Done false
22384 6609 Acked Read 2 Sent Done Done false
22385 6610 Acked Flush 0 Sent Done Done false
22386 6611 Acked Read 3 Sent Done Done false
22387 6612 Acked Read 3 Sent Done Done false
22388 6613 Acked Write 10 Sent Done Done false
22389 6614 Acked Write 5 Sent Done Done false
22390 6615 Acked Flush 0 Sent Done Done false
22391 6616 Acked Read 7 Sent Done Done false
22392 6617 Acked Read 3 Sent Done Done false
22393 6618 Acked Read 1 Sent Done Done false
22394 6619 Acked Write 6 Sent Done Done false
22395 6620 Acked Write 8 Sent Done Done false
22396 6621 Acked Read 1 Sent Done Done false
22397 6622 Acked Read 4 Sent Done Done false
22398 6623 Acked Write 3 Sent Done Done false
22399 6624 Acked Read 8 Sent Done Done false
22400 6625 Acked Write 5 Sent Done Done false
22401 6626 Acked Read 10 Sent Done Done false
22402 6627 Acked Read 3 Sent Done Done false
22403 6628 Acked Read 4 Sent Done Done false
22404 6629 Acked Read 5 Sent Done Done false
22405 6630 Acked Write 2 Sent Done Done false
22406 6631 Acked Read 2 Sent Done Done false
22407 6632 Acked Write 3 Sent Done Done false
22408 6633 Acked Read 5 Sent Done Done false
22409 6634 Acked Read 9 Sent Done Done false
22410 6635 Acked Read 10 Sent Done Done false
22411 6636 Acked Read 5 Sent Done Done false
22412 6637 Acked Read 7 Sent Done Done false
22413 6638 Acked Write 8 Sent Done Done false
22414 6639 Acked Read 8 Sent Done Done false
22415 6640 Acked Write 9 Sent Done Done false
22416 6641 Acked Read 5 Sent Done Done false
22417 6642 Acked Read 7 Sent Done Done false
22418 6643 Acked Read 6 Sent Done Done false
22419 6644 Acked Read 6 Sent Done Done false
22420 6645 Acked Read 9 Sent Done Done false
22421 6646 Acked Read 8 Sent Done Done false
22422 6647 Acked Write 2 Sent Done Done false
22423 6648 Acked Flush 0 Sent Done Done false
22424 6649 Acked Write 6 Sent Done Done false
22425 6650 Acked Write 3 Sent Done Done false
22426 6651 Acked Read 5 Sent Done Done false
22427 6652 Acked Write 1 Sent Done Done false
22428 6653 Acked Read 7 Sent Done Done false
22429 6654 Acked Write 5 Sent Done Done false
22430 6655 Acked Write 3 Sent Done Done false
22431 6656 Acked Flush 0 Sent Done Done false
22432 6657 Acked Write 2 Sent Done Done false
22433 6658 Acked Read 10 Sent Done Done false
22434 6659 Acked Flush 0 Sent Done Done false
22435 6660 Acked Read 3 Sent Done Done false
22436 6661 Acked Write 5 Sent Done Done false
22437 6662 Acked Write 4 Sent Done Done false
22438 6663 Acked Read 5 Sent Done Done false
22439 6664 Acked Write 5 Sent Done Done false
22440 6665 Acked Read 7 Sent Done Done false
22441 6666 Acked Read 5 Sent Done Done false
22442 6667 Acked Write 10 Sent Done Done false
22443 6668 Acked Read 4 Sent Done Done false
22444 6669 Acked Read 1 Sent Done Done false
22445 6670 Acked Flush 0 Sent Done Done false
22446 6671 Acked Write 10 Sent Done Done false
22447 6672 Acked Read 10 Sent Done Done false
22448 6673 Acked Read 9 Sent Done Done false
22449 6674 Acked Write 7 Sent Done Done false
22450 6675 Acked Flush 0 Sent Done Done false
22451 6676 Acked Read 3 Sent Done Done false
22452 6677 Acked Write 8 Sent Done Done false
22453 6678 Acked Flush 0 Sent Done Done false
22454 6679 Acked Flush 0 Sent Done Done false
22455 6680 Acked Write 1 Sent Done Done false
22456 6681 Acked Write 3 Sent Done Done false
22457 6682 Acked Write 8 Sent Done Done false
22458 6683 Acked Read 7 Sent Done Done false
22459 6684 Acked Read 2 Sent Done Done false
22460 6685 Acked Write 6 Sent Done Done false
22461 6686 Acked Read 9 Sent Done Done false
22462 6687 Acked Write 1 Sent Done Done false
22463 6688 Acked Write 7 Sent Done Done false
22464 6689 Acked Read 8 Sent Done Done false
22465 6690 Acked Flush 0 Sent Done Done false
22466 6691 Acked Read 9 Sent Done Done false
22467 6692 Acked Read 8 Sent Done Done false
22468 6693 Acked Read 9 Sent Done Done false
22469 6694 Acked Read 1 Sent Done Done false
22470 6695 Acked Write 5 Sent Done Done false
22471 6696 Acked Flush 0 Sent Done Done false
22472 6697 Acked Write 2 Sent Done Done false
22473 6698 Acked Read 9 Sent Done Done false
22474 6699 Acked Read 7 Sent Done Done false
22475 6700 Acked Write 10 Sent Done Done false
22476 6701 Acked Read 10 Sent Done Done false
22477 6702 Acked Read 7 Sent Done Done false
22478 6703 Acked Read 1 Sent Done Done false
22479 6704 Acked Read 2 Sent Done Done false
22480 6705 Acked Read 5 Sent Done Done false
22481 6706 Acked Read 4 Sent Done Done false
22482 6707 Acked Read 10 Sent Done Done false
22483 6708 Acked Flush 0 Sent Done Done false
22484 6709 Acked Write 8 Sent Done Done false
22485 6710 Acked Read 10 Sent Done Done false
22486 6711 Acked Write 3 Sent Done Done false
22487 6712 Acked Write 10 Sent Done Done false
22488 6713 Acked Write 4 Sent Done Done false
22489 6714 Acked Read 3 Sent Done Done false
22490 6715 Acked Write 4 Sent Done Done false
22491 6716 Acked Read 9 Sent Done Done false
22492 6717 Acked Flush 0 Sent Done Done false
22493 6718 Acked Read 2 Sent Done Done false
22494 6719 Acked Read 6 Sent Done Done false
22495 6720 Acked Read 2 Sent Done Done false
22496 6721 Acked Flush 0 Sent Done Done false
22497 6722 Acked Read 10 Sent Done Done false
22498 6723 Acked Read 2 Sent Done Done false
22499 6724 Acked Read 4 Sent Done Done false
22500 6725 Acked Read 6 Sent Done Done false
22501 6726 Acked Read 5 Sent Done Done false
22502 6727 Acked Write 8 Sent Done Done false
22503 6728 Acked Flush 0 Sent Done Done false
22504 6729 Acked Write 1 Sent Done Done false
22505 6730 Acked Read 1 Sent Done Done false
22506 6731 Acked Write 4 Sent Done Done false
22507 6732 Acked Write 4 Sent Done Done false
22508 6733 Acked Read 9 Sent Done Done false
22509 6734 Acked Write 10 Sent Done Done false
22510 6735 Acked Flush 0 Sent Done Done false
22511 6736 Acked Read 5 Sent Done Done false
22512 6737 Acked Read 1 Sent Done Done false
22513 6738 Acked Read 7 Sent Done Done false
22514 6739 Acked Write 8 Sent Done Done false
22515 6740 Acked Read 4 Sent Done Done false
22516 6741 Acked Write 3 Sent Done Done false
22517 6742 Acked Write 3 Sent Done Done false
22518 6743 Acked Read 8 Sent Done Done false
22519 6744 Acked Flush 0 Sent Done Done false
22520 STATES DS:0 DS:1 DS:2 TOTAL
22521 Sent 303 0 0 303
22522 Done 0 303 303 606
22523 Skipped 0 0 0 0
22524 Error 0 0 0 0
22525 Last Flush: 6441 6744 6744
22526 Downstairs last five completed: 6441 6440 6439 6438 6437
22527 Upstairs last five completed: 6744 6743 6742 6741 6740
22528 CLIENT: Up:0 ds:303 act:2
225292024-11-07T14:08:01.144ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225302024-11-07T14:08:01.144ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37745) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225312024-11-07T14:08:01.145ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225322024-11-07T14:08:01.146ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225332024-11-07T14:08:01.146ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225342024-11-07T14:08:01.146ZINFOcrucible: send last flush ID to this DS: 6441 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225352024-11-07T14:08:01.146ZINFOcrucible: Replied this last flush ID: 6441 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225362024-11-07T14:08:01.146ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225372024-11-07T14:08:01.146ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225382024-11-07T14:08:01.146ZINFOcrucible: [0] client re-new 303 jobs since flush 6441 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225392024-11-07T14:08:01.146ZINFOcrucible: [0] Replayed 303 jobs since flush: 6441 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
22540 ----------------------------------------------------------------
22541 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22542 Downstairs last five completed: 6744 6743 6742 6741 6740
22543 Upstairs last five completed: 6744 6743 6742 6741 6740
22544 CLIENT: Up:0 ds:0 act:3
22545 Replay: All jobs finished
22546 Main task has work to do, go find it
22547 got dsc Stop(0)
22548 stop 0
22549 Nov 07 14:08:03.446 INFO request completed, latency_us: [8810] Got stop action so:false kr:false
22550 348, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 1a628407-1beb-40dc-bb48-4997f4381b31, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
225512024-11-07T14:08:03.446ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225522024-11-07T14:08:03.446ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225532024-11-07T14:08:03.446ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225542024-11-07T14:08:03.446ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
225552024-11-07T14:08:03.446ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
22556 Nov 07 14:08:03.446 INFO request completed, latency_us: 156, response_code: 200, uri: /state/cid/0, method: GET, req_id: 7052cb13-2050-4df8-bec1-0fa9c9775208, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:08:03.446982005Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"":"io task","client":"0","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
22557 :9998
22558 [8810] Exited with: ExitStatus(unix_wait_status(9))
22559 [8810][0] reports Exit
22560 Nov 07 14:08:07.448 INFO request completed, latency_us: 238, response_code: 200, uri: /state/cid/0, method: GET, req_id: 54754571-e0f2-49e9-bcc2-2adc99245023, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
22561 001/300 Write block 3722 len 20480 data: 9 7 6 5 4
22562 002/300 Read block 1107 len 20480
22563 003/300 Read block 3686 len 16384
22564 004/300 Write block 2284 len 36864 data: 3 3 3 2 2 2 2 2 2
22565 005/300 Read block 4947 len 16384
22566 006/300 Read block 1941 len 20480
22567 007/300 Write block 3891 len 4096 data: 5
22568 008/300 Write block 2770 len 24576 data: 3 2 2 2 3 3
22569 009/300 Read block 4245 len 16384
22570 010/300 Read block 2954 len 20480
22571 011/300 Write block 550 len 12288 data: 5 5 5
22572 012/300 Write block 4864 len 20480 data: 4 2 2 3 3
22573 013/300 Write block 619 len 20480 data: 3 4 4 5 5
22574 014/300 Write block 379 len 8192 data: 6 8
22575 015/300 Read block 910 len 36864
22576 016/300 Read block 2936 len 40960
22577 017/300 Read block 783 len 12288
22578 018/300 Write block 2227 len 20480 data: 4 5 5 5 5
22579 019/300 Write block 50 len 8192 data: 5 4
22580 020/300 Read block 2232 len 32768
22581 021/300 Write block 2249 len 24576 data: 4 3 4 4 3 3
22582 022/300 Write block 4655 len 32768 data: 2 2 2 3 3 3 4 4
22583 023/300 Read block 3521 len 24576
22584 024/300 Read block 3497 len 8192
22585 025/300 Write block 1934 len 40960 data: 5 4 3 3 3 2 4 4 4 6
22586 026/300 Write block 1058 len 36864 data: 2 3 3 4 4 3 2 2 3
22587 027/300 Read block 98 len 12288
22588 028/300 Write block 321 len 28672 data: 3 2 2 4 5 4 4
22589 029/300 Write block 1351 len 36864 data: 3 2 5 7 6 9 9 9 10
22590 030/300 Read block 4404 len 36864
22591 031/300 Read block 3098 len 24576
22592 032/300 Write block 0 len 16384 data: 3 4 5 7
22593 033/300 Read block 1995 len 32768
22594 034/300 Read block 1813 len 8192
22595 035/300 Read block 3294 len 4096
22596 036/300 Read block 862 len 16384
22597 037/300 Write block 1972 len 32768 data: 4 4 4 2 2 2 4 5
22598 038/300 Read block 4041 len 4096
22599 039/300 Write block 1333 len 40960 data: 10 8 8 7 7 6 4 4 4 5
22600 040/300 Write block 198 len 24576 data: 4 3 4 3 3 3
22601 041/300 Read block 2187 len 24576
22602 042/300 Read block 4280 len 16384
22603 043/300 Write block 4685 len 16384 data: 3 3 5 4
22604 044/300 Read block 4101 len 28672
22605 045/300 Write block 3083 len 12288 data: 7 7 6
22606 046/300 Read block 2060 len 36864
22607 047/300 Read block 2895 len 4096
22608 048/300 Write block 689 len 12288 data: 3 3 2
22609 049/300 Write block 1523 len 12288 data: 3 3 4
22610 050/300 Write block 1635 len 32768 data: 4 5 5 5 5 5 4 4
22611 051/300 Write block 709 len 20480 data: 7 6 5 5 5
22612 052/300 Flush
22613 053/300 Write block 661 len 4096 data: 3
22614 054/300 Read block 1340 len 32768
22615 055/300 Read block 3122 len 20480
22616 056/300 Write block 612 len 24576 data: 4 4 4 3 3 2
22617 057/300 Write block 2101 len 24576 data: 3 3 3 5 4 5
22618 058/300 Write block 1840 len 32768 data: 4 5 5 7 6 5 5 5
22619 059/300 Read block 4290 len 32768
22620 060/300 Write block 1764 len 16384 data: 4 5 3 4
22621 061/300 Read block 829 len 12288
22622 062/300 Write block 4102 len 16384 data: 5 6 6 7
22623 063/300 Write block 3112 len 16384 data: 3 3 4 5
22624 064/300 Write block 709 len 20480 data: 8 7 6 6 6
22625 065/300 Read block 2033 len 20480
22626 066/300 Write block 189 len 8192 data: 5 6
22627 067/300 Read block 3581 len 24576
22628 068/300 Flush
22629 069/300 Write block 796 len 16384 data: 5 4 3 4
22630 070/300 Write block 4060 len 24576 data: 4 5 5 4 4 4
22631 071/300 Read block 655 len 8192
22632 072/300 Write block 2007 len 40960 data: 6 7 6 6 6 7 8 7 6 5
22633 073/300 Read block 2172 len 8192
22634 074/300 Read block 1597 len 24576
22635 075/300 Flush
22636 076/300 Write block 816 len 16384 data: 2 3 3 3
22637 077/300 Flush
22638 078/300 Read block 708 len 20480
22639 079/300 Read block 3366 len 36864
22640 080/300 Write block 3545 len 36864 data: 7 7 6 7 6 5 6 8 7
22641 081/300 Read block 2859 len 36864
22642 082/300 Flush
22643 083/300 Write block 277 len 40960 data: 5 3 3 2 2 2 2 2 3 4
22644 084/300 Read block 713 len 32768
22645 085/300 Write block 1908 len 36864 data: 3 3 3 2 2 2 3 3 3
22646 086/300 Write block 3486 len 12288 data: 6 6 7
22647 087/300 Read block 2482 len 32768
22648 088/300 Write block 2152 len 20480 data: 3 3 2 2 2
22649 089/300 Read block 137 len 4096
22650 090/300 Read block 1224 len 16384
22651 091/300 Read block 4997 len 8192
22652 092/300 Read block 1317 len 28672
22653 093/300 Read block 3556 len 40960
22654 094/300 Read block 1487 len 8192
22655 095/300 Flush
22656 096/300 Read block 3582 len 20480
22657 097/300 Read block 2195 len 28672
22658 098/300 Write block 1942 len 8192 data: 5 7
22659 099/300 Flush
22660 100/300 Write block 3264 len 28672 data: 4 5 5 7 4 6 6
22661 101/300 Read block 1760 len 24576
22662 102/300 Flush
22663 103/300 Read block 1452 len 12288
22664 104/300 Flush
22665 105/300 Read block 2615 len 20480
22666 106/300 Flush
22667 107/300 Read block 942 len 12288
22668 108/300 Read block 316 len 16384
22669 109/300 Write block 4935 len 36864 data: 4 3 2 3 3 2 2 2 2
22670 110/300 Flush
22671 111/300 Read block 960 len 16384
22672 112/300 Write block 3488 len 4096 data: 8
22673 113/300 Read block 3309 len 16384
22674 114/300 Write block 1963 len 32768 data: 5 3 3 4 5 4 4 4
22675 115/300 Write block 487 len 8192 data: 6 6
22676 116/300 Read block 848 len 28672
22677 117/300 Read block 1039 len 12288
22678 118/300 Write block 4224 len 36864 data: 5 6 6 6 7 7 7 6 6
22679 119/300 Write block 4647 len 24576 data: 7 7 5 3 3 2
22680 120/300 Write block 2722 len 12288 data: 5 5 5
22681 121/300 Read block 2934 len 28672
22682 122/300 Write block 1325 len 28672 data: 6 5 6 7 8 9 8
22683 123/300 Write block 2847 len 24576 data: 5 5 5 5 4 6
22684 124/300 Write block 1066 len 4096 data: 4
22685 125/300 Read block 85 len 16384
22686 126/300 Read block 2643 len 40960
22687 127/300 Write block 494 len 32768 data: 3 3 3 4 3 3 3 3
22688 128/300 Read block 1889 len 8192
22689 129/300 Read block 924 len 28672
22690 130/300 Read block 3726 len 20480
22691 131/300 Read block 1473 len 20480
22692 132/300 Read block 341 len 20480
22693 133/300 Read block 1055 len 4096
22694 134/300 Read block 425 len 32768
22695 135/300 Read block 4727 len 8192
22696 136/300 Flush
22697 137/300 Read block 4070 len 12288
22698 138/300 Read block 1247 len 8192
22699 139/300 Write block 1204 len 4096 data: 3
22700 140/300 Read block 2643 len 8192
22701 141/300 Read block 4264 len 40960
22702 142/300 Write block 3388 len 40960 data: 4 3 3 3 3 4 6 6 5 4
22703 143/300 Read block 1301 len 12288
22704 144/300 Read block 4330 len 32768
22705 145/300 Write block 2078 len 8192 data: 4 5
22706 146/300 Write block 1519 len 12288 data: 3 2 3
22707 147/300 Write block 3674 len 24576 data: 4 4 3 4 3 3
22708 148/300 Flush
22709 149/300 Read block 3682 len 12288
22710 150/300 Read block 563 len 4096
22711 151/300 Read block 4328 len 8192
22712 152/300 Read block 2 len 12288
22713 153/300 Read block 3637 len 40960
22714 154/300 Read block 4177 len 32768
22715 155/300 Write block 593 len 8192 data: 4 4
22716 156/300 Write block 541 len 4096 data: 3
22717 157/300 Read block 3004 len 20480
22718 158/300 Write block 4007 len 32768 data: 6 5 6 7 7 8 6 6
22719 159/300 Read block 3351 len 12288
22720 160/300 Write block 4872 len 28672 data: 3 3 3 3 3 3 2
22721 161/300 Read block 1688 len 24576
22722 162/300 Read block 4454 len 32768
22723 163/300 Flush
22724 164/300 Read block 371 len 20480
22725 165/300 Write block 2145 len 32768 data: 5 6 3 3 3 4 4 4
22726 166/300 Read block 2803 len 36864
22727 167/300 Read block 2238 len 40960
22728 168/300 Flush
22729 169/300 Write block 415 len 36864 data: 4 4 4 3 3 4 3 4 4
22730 170/300 Read block 2471 len 16384
22731 171/300 Flush
22732 172/300 Write block 3080 len 8192 data: 4 7
22733 173/300 Write block 2147 len 36864 data: 4 4 4 5 5 5 4 3 3
22734 174/300 Read block 1316 len 20480
22735 175/300 Write block 2532 len 8192 data: 4 3
22736 176/300 Flush
22737 177/300 Read block 4784 len 8192
22738 178/300 Read block 3986 len 32768
22739 179/300 Read block 2706 len 24576
22740 180/300 Flush
22741 181/300 Read block 1708 len 24576
22742 182/300 Write block 3693 len 32768 data: 7 9 9 9 8 9 8 6
22743 183/300 Read block 3897 len 40960
22744 184/300 Write block 1598 len 16384 data: 5 6 7 8
22745 185/300 Write block 4417 len 24576 data: 5 4 4 4 5 4
22746 186/300 Write block 657 len 20480 data: 3 3 3 3 4
22747 187/300 Read block 2822 len 16384
22748 188/300 Write block 3777 len 8192 data: 3 3
22749 189/300 Write block 2592 len 24576 data: 6 7 7 7 7 6
22750 190/300 Write block 3882 len 36864 data: 6 8 7 5 5 5 5 6 6
22751 191/300 Write block 1680 len 12288 data: 5 6 5
22752 192/300 Read block 1890 len 36864
22753 193/300 Read block 4444 len 32768
22754 194/300 Write block 1913 len 36864 data: 3 4 4 4 3 3 5 5 5
22755 195/300 Read block 4711 len 12288
22756 196/300 Write block 1658 len 16384 data: 4 6 7 7
22757 197/300 Read block 316 len 16384
22758 198/300 Read block 4784 len 20480
22759 199/300 Write block 3630 len 16384 data: 9 7 7 6
22760 200/300 Read block 2915 len 40960
22761 201/300 Write block 4034 len 28672 data: 3 4 4 4 4 3 3
22762 202/300 Write block 191 len 36864 data: 4 5 4 4 4 4 4 5 4
22763 203/300 Read block 4864 len 28672
22764 204/300 Write block 4286 len 12288 data: 3 3 5
22765 205/300 Write block 4669 len 40960 data: 5 4 4 6 5 5 5 4 4 4
22766 206/300 Write block 1946 len 32768 data: 7 5 6 6 7 5 4 3
22767 207/300 Write block 2150 len 12288 data: 6 6 6
22768 208/300 Read block 2479 len 40960
22769 209/300 Read block 4683 len 4096
22770 210/300 Read block 447 len 4096
22771 211/300 Flush
22772 212/300 Write block 1985 len 12288 data: 6 7 7
22773 213/300 Flush
22774 214/300 Read block 4405 len 24576
22775 215/300 Write block 3054 len 4096 data: 6
22776 216/300 Write block 3967 len 24576 data: 7 7 6 5 5 7
22777 217/300 Read block 416 len 16384
22778 218/300 Read block 3008 len 16384
22779 219/300 Flush
22780 220/300 Flush
22781 221/300 Read block 999 len 12288
22782 222/300 Write block 2378 len 28672 data: 3 3 4 3 3 3 4
22783 223/300 Write block 996 len 20480 data: 4 3 3 3 4
22784 224/300 Write block 2853 len 12288 data: 7 8 5
22785 225/300 Write block 4526 len 20480 data: 7 7 8 6 7
22786 226/300 Read block 374 len 36864
22787 227/300 Read block 2028 len 12288
22788 228/300 Flush
22789 229/300 Read block 4635 len 16384
22790 230/300 Read block 3535 len 12288
22791 231/300 Flush
22792 232/300 Read block 4268 len 40960
22793 233/300 Write block 792 len 40960 data: 5 5 7 7 6 5 4 5 3 3
22794 234/300 Read block 3336 len 40960
22795 235/300 Read block 1909 len 4096
22796 236/300 Flush
22797 237/300 Write block 3 len 16384 data: 8 7 6 6
22798 238/300 Read block 3660 len 28672
22799 239/300 Write block 4249 len 12288 data: 3 4 5
22800 240/300 Read block 514 len 4096
22801 241/300 Write block 4307 len 28672 data: 5 4 6 5 7 6 6
22802 242/300 Write block 2477 len 12288 data: 6 6 5
22803 243/300 Write block 2604 len 28672 data: 4 3 3 3 3 4 4
22804 244/300 Write block 1318 len 36864 data: 4 4 4 4 4 2 4 7 6
22805 245/300 Write block 2433 len 28672 data: 3 3 3 3 5 6 6
22806 246/300 Read block 4236 len 12288
22807 247/300 Write block 3686 len 32768 data: 6 4 3 2 3 4 6 8
22808 248/300 Write block 4357 len 28672 data: 4 3 3 4 4 6 5
22809 249/300 Write block 139 len 16384 data: 2 3 4 5
22810 250/300 Read block 524 len 20480
22811 251/300 Read block 1398 len 16384
22812 252/300 Flush
22813 253/300 Flush
22814 254/300 Read block 4750 len 4096
22815 255/300 Write block 2742 len 12288 data: 4 4 4
22816 256/300 Flush
22817 257/300 Write block 3493 len 4096 data: 5
22818 258/300 Write block 2906 len 16384 data: 4 4 4 5
22819 259/300 Write block 3738 len 32768 data: 6 7 5 5 4 5 5 5
22820 260/300 Write block 2695 len 20480 data: 4 5 5 5 5
22821 261/300 Write block 2339 len 20480 data: 4 4 3 4 4
22822 262/300 Read block 4634 len 40960
22823 263/300 Flush
22824 264/300 Read block 3368 len 20480
22825 265/300 Write block 1764 len 24576 data: 5 6 4 5 5 7
22826 266/300 Write block 2291 len 24576 data: 3 3 3 3 3 3
22827 267/300 Read block 67 len 12288
22828 268/300 Write block 1648 len 40960 data: 8 7 6 6 6 6 6 5 5 6
22829 269/300 Write block 1261 len 28672 data: 4 3 2 4 5 5 5
22830 270/300 Write block 384 len 36864 data: 4 5 5 5 3 3 3 3 3
22831 271/300 Read block 4397 len 12288
22832 272/300 Write block 916 len 28672 data: 6 6 6 7 4 3 3
22833 273/300 Read block 3839 len 12288
22834 274/300 Read block 1257 len 36864
22835 275/300 Write block 593 len 4096 data: 5
22836 276/300 Read block 2948 len 36864
22837 277/300 Write block 2796 len 32768 data: 5 4 4 3 3 5 4 5
22838 278/300 Write block 2357 len 40960 data: 3 3 4 4 4 4 4 4 6 4
22839 279/300 Read block 374 len 8192
22840 280/300 Write block 3789 len 20480 data: 5 5 5 5 5
22841 281/300 Read block 2466 len 32768
22842 282/300 Write block 2415 len 36864 data: 3 3 4 5 7 7 7 5 6
22843 283/300 Read block 1955 len 12288
22844 284/300 Read block 883 len 4096
22845 285/300 Read block 2055 len 36864
22846 286/300 Read block 435 len 24576
22847 287/300 Read block 1521 len 32768
22848 288/300 Write block 542 len 16384 data: 3 5 5 5
22849 289/300 Write block 4021 len 40960 data: 6 6 5 4 2 2 2 2 2 2
22850 290/300 Flush
22851 291/300 Flush
22852 292/300 Read block 4059 len 36864
22853 293/300 Write block 1253 len 12288 data: 4 5 4
22854 294/300 Read block 1224 len 8192
22855 295/300 Write block 4953 len 12288 data: 7 8 9
22856 296/300 Read block 1375 len 4096
22857 297/300 Read block 1929 len 36864
22858 298/300 Write block 3381 len 32768 data: 7 6 5 5 5 4 5 5
22859 299/300 Write block 1711 len 16384 data: 7 6 6 6
22860 300/300 Read block 3680 len 40960
22861 301/300 Read block 3110 len 8192
22862 Main task has work to do, go find it
22863 got dsc Start(0)
22864 start 0
22865 Nov 07 14:08:07.726 INFO request completed, latency_us: 197, response_code: 204, uri: /start/cid/0, method: POST, req_id: 6816c78d-87a5-45bf-8a29-7368b112152c, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
22866 Replay: started 0, returned:Ok(())
22867 ----------------------------------------------------------------
22868 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22869 States: Offline Active Active
22870 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22871 6745 Acked Write 5 Sent Done Done false
22872 6746 Acked Read 5 Sent Done Done false
22873 6747 Acked Read 4 Sent Done Done false
22874 6748 Acked Write 9 Sent Done Done false
22875 6749 Acked Read 4 Sent Done Done false
22876 6750 Acked Read 5 Sent Done Done false
22877 6751 Acked Write 1 Sent Done Done false
22878 6752 Acked Write 6 Sent Done Done false
22879 6753 Acked Read 4 Sent Done Done false
22880 6754 Acked Read 5 Sent Done Done false
22881 6755 Acked Write 3 Sent Done Done false
22882 6756 Acked Write 5 Sent Done Done false
22883 6757 Acked Write 5 Sent Done Done false
22884 6758 Acked Write 2 Sent Done Done false
22885 6759 Acked Read 9 Sent Done Done false
22886 6760 Acked Read 10 Sent Done Done false
22887 6761 Acked Read 3 Sent Done Done false
22888 6762 Acked Write 5 Sent Done Done false
22889 6763 Acked Write 2 Sent Done Done false
22890 6764 Acked Read 8 Sent Done Done false
22891 6765 Acked Write 6 Sent Done Done false
22892 6766 Acked Write 8 Sent Done Done false
22893 6767 Acked Read 6 Sent Done Done false
22894 6768 Acked Read 2 Sent Done Done false
22895 6769 Acked Write 10 Sent Done Done false
22896 6770 Acked Write 9 Sent Done Done false
22897 6771 Acked Read 3 Sent Done Done false
22898 6772 Acked Write 7 Sent Done Done false
22899 6773 Acked Write 9 Sent Done Done false
22900 6774 Acked Read 9 Sent Done Done false
22901 6775 Acked Read 6 Sent Done Done false
22902 6776 Acked Write 4 Sent Done Done false
22903 6777 Acked Read 8 Sent Done Done false
22904 6778 Acked Read 2 Sent Done Done false
22905 6779 Acked Read 1 Sent Done Done false
22906 6780 Acked Read 4 Sent Done Done false
22907 6781 Acked Write 8 Sent Done Done false
22908 6782 Acked Read 1 Sent Done Done false
22909 6783 Acked Write 10 Sent Done Done false
22910 6784 Acked Write 6 Sent Done Done false
22911 6785 Acked Read 6 Sent Done Done false
22912 6786 Acked Read 4 Sent Done Done false
22913 6787 Acked Write 4 Sent Done Done false
22914 6788 Acked Read 7 Sent Done Done false
22915 6789 Acked Write 3 Sent Done Done false
22916 6790 Acked Read 9 Sent Done Done false
22917 6791 Acked Read 1 Sent Done Done false
22918 6792 Acked Write 3 Sent Done Done false
22919 6793 Acked Write 3 Sent Done Done false
22920 6794 Acked Write 8 Sent Done Done false
22921 6795 Acked Write 5 Sent Done Done false
22922 6796 Acked Flush 0 Sent Done Done false
22923 6797 Acked Write 1 Sent Done Done false
22924 6798 Acked Read 8 Sent Done Done false
22925 6799 Acked Read 5 Sent Done Done false
22926 6800 Acked Write 6 Sent Done Done false
22927 6801 Acked Write 6 Sent Done Done false
22928 6802 Acked Write 8 Sent Done Done false
22929 6803 Acked Read 8 Sent Done Done false
22930 6804 Acked Write 4 Sent Done Done false
22931 6805 Acked Read 3 Sent Done Done false
22932 6806 Acked Write 4 Sent Done Done false
22933 6807 Acked Write 4 Sent Done Done false
22934 6808 Acked Write 5 Sent Done Done false
22935 6809 Acked Read 5 Sent Done Done false
22936 6810 Acked Write 2 Sent Done Done false
22937 6811 Acked Read 6 Sent Done Done false
22938 6812 Acked Flush 0 Sent Done Done false
22939 6813 Acked Write 4 Sent Done Done false
22940 6814 Acked Write 6 Sent Done Done false
22941 6815 Acked Read 2 Sent Done Done false
22942 6816 Acked Write 10 Sent Done Done false
22943 6817 Acked Read 2 Sent Done Done false
22944 6818 Acked Read 6 Sent Done Done false
22945 6819 Acked Flush 0 Sent Done Done false
22946 6820 Acked Write 4 Sent Done Done false
22947 6821 Acked Flush 0 Sent Done Done false
22948 6822 Acked Read 5 Sent Done Done false
22949 6823 Acked Read 9 Sent Done Done false
22950 6824 Acked Write 9 Sent Done Done false
22951 6825 Acked Read 9 Sent Done Done false
22952 6826 Acked Flush 0 Sent Done Done false
22953 6827 Acked Write 10 Sent Done Done false
22954 6828 Acked Read 8 Sent Done Done false
22955 6829 Acked Write 9 Sent Done Done false
22956 6830 Acked Write 3 Sent Done Done false
22957 6831 Acked Read 8 Sent Done Done false
22958 6832 Acked Write 5 Sent Done Done false
22959 6833 Acked Read 1 Sent Done Done false
22960 6834 Acked Read 4 Sent Done Done false
22961 6835 Acked Read 2 Sent Done Done false
22962 6836 Acked Read 7 Sent Done Done false
22963 6837 Acked Read 10 Sent Done Done false
22964 6838 Acked Read 2 Sent Done Done false
22965 6839 Acked Flush 0 Sent Done Done false
22966 6840 Acked Read 5 Sent Done Done false
22967 6841 Acked Read 7 Sent Done Done false
22968 6842 Acked Write 2 Sent Done Done false
22969 6843 Acked Flush 0 Sent Done Done false
22970 6844 Acked Write 7 Sent Done Done false
22971 6845 Acked Read 6 Sent Done Done false
22972 6846 Acked Flush 0 Sent Done Done false
22973 6847 Acked Read 3 Sent Done Done false
22974 6848 Acked Flush 0 Sent Done Done false
22975 6849 Acked Read 5 Sent Done Done false
22976 6850 Acked Flush 0 Sent Done Done false
22977 6851 Acked Read 3 Sent Done Done false
22978 6852 Acked Read 4 Sent Done Done false
22979 6853 Acked Write 9 Sent Done Done false
22980 6854 Acked Flush 0 Sent Done Done false
22981 6855 Acked Read 4 Sent Done Done false
22982 6856 Acked Write 1 Sent Done Done false
22983 6857 Acked Read 4 Sent Done Done false
22984 6858 Acked Write 8 Sent Done Done false
22985 6859 Acked Write 2 Sent Done Done false
22986 6860 Acked Read 7 Sent Done Done false
22987 6861 Acked Read 3 Sent Done Done false
22988 6862 Acked Write 9 Sent Done Done false
22989 6863 Acked Write 6 Sent Done Done false
22990 6864 Acked Write 3 Sent Done Done false
22991 6865 Acked Read 7 Sent Done Done false
22992 6866 Acked Write 7 Sent Done Done false
22993 6867 Acked Write 6 Sent Done Done false
22994 6868 Acked Write 1 Sent Done Done false
22995 6869 Acked Read 4 Sent Done Done false
22996 6870 Acked Read 10 Sent Done Done false
22997 6871 Acked Write 8 Sent Done Done false
22998 6872 Acked Read 2 Sent Done Done false
22999 6873 Acked Read 7 Sent Done Done false
23000 6874 Acked Read 5 Sent Done Done false
23001 6875 Acked Read 5 Sent Done Done false
23002 6876 Acked Read 5 Sent Done Done false
23003 6877 Acked Read 1 Sent Done Done false
23004 6878 Acked Read 8 Sent Done Done false
23005 6879 Acked Read 2 Sent Done Done false
23006 6880 Acked Flush 0 Sent Done Done false
23007 6881 Acked Read 3 Sent Done Done false
23008 6882 Acked Read 2 Sent Done Done false
23009 6883 Acked Write 1 Sent Done Done false
23010 6884 Acked Read 2 Sent Done Done false
23011 6885 Acked Read 10 Sent Done Done false
23012 6886 Acked Write 10 Sent Done Done false
23013 6887 Acked Read 3 Sent Done Done false
23014 6888 Acked Read 8 Sent Done Done false
23015 6889 Acked Write 2 Sent Done Done false
23016 6890 Acked Write 3 Sent Done Done false
23017 6891 Acked Write 6 Sent Done Done false
23018 6892 Acked Flush 0 Sent Done Done false
23019 6893 Acked Read 3 Sent Done Done false
23020 6894 Acked Read 1 Sent Done Done false
23021 6895 Acked Read 2 Sent Done Done false
23022 6896 Acked Read 3 Sent Done Done false
23023 6897 Acked Read 10 Sent Done Done false
23024 6898 Acked Read 8 Sent Done Done false
23025 6899 Acked Write 2 Sent Done Done false
23026 6900 Acked Write 1 Sent Done Done false
23027 6901 Acked Read 5 Sent Done Done false
23028 6902 Acked Write 8 Sent Done Done false
23029 6903 Acked Read 3 Sent Done Done false
23030 6904 Acked Write 7 Sent Done Done false
23031 6905 Acked Read 6 Sent Done Done false
23032 6906 Acked Read 8 Sent Done Done false
23033 6907 Acked Flush 0 Sent Done Done false
23034 6908 Acked Read 5 Sent Done Done false
23035 6909 Acked Write 8 Sent Done Done false
23036 6910 Acked Read 9 Sent Done Done false
23037 6911 Acked Read 10 Sent Done Done false
23038 6912 Acked Flush 0 Sent Done Done false
23039 6913 Acked Write 9 Sent Done Done false
23040 6914 Acked Read 4 Sent Done Done false
23041 6915 Acked Flush 0 Sent Done Done false
23042 6916 Acked Write 2 Sent Done Done false
23043 6917 Acked Write 9 Sent Done Done false
23044 6918 Acked Read 5 Sent Done Done false
23045 6919 Acked Write 2 Sent Done Done false
23046 6920 Acked Flush 0 Sent Done Done false
23047 6921 Acked Read 2 Sent Done Done false
23048 6922 Acked Read 8 Sent Done Done false
23049 6923 Acked Read 6 Sent Done Done false
23050 6924 Acked Flush 0 Sent Done Done false
23051 6925 Acked Read 6 Sent Done Done false
23052 6926 Acked Write 8 Sent Done Done false
23053 6927 Acked Read 10 Sent Done Done false
23054 6928 Acked Write 4 Sent Done Done false
23055 6929 Acked Write 6 Sent Done Done false
23056 6930 Acked Write 5 Sent Done Done false
23057 6931 Acked Read 4 Sent Done Done false
23058 6932 Acked Write 2 Sent Done Done false
23059 6933 Acked Write 6 Sent Done Done false
23060 6934 Acked Write 9 Sent Done Done false
23061 6935 Acked Write 3 Sent Done Done false
23062 6936 Acked Read 9 Sent Done Done false
23063 6937 Acked Read 8 Sent Done Done false
23064 6938 Acked Write 9 Sent Done Done false
23065 6939 Acked Read 3 Sent Done Done false
23066 6940 Acked Write 4 Sent Done Done false
23067 6941 Acked Read 4 Sent Done Done false
23068 6942 Acked Read 5 Sent Done Done false
23069 6943 Acked Write 4 Sent Done Done false
23070 6944 Acked Read 10 Sent Done Done false
23071 6945 Acked Write 7 Sent Done Done false
23072 6946 Acked Write 9 Sent Done Done false
23073 6947 Acked Read 7 Sent Done Done false
23074 6948 Acked Write 3 Sent Done Done false
23075 6949 Acked Write 10 Sent Done Done false
23076 6950 Acked Write 8 Sent Done Done false
23077 6951 Acked Write 3 Sent Done Done false
23078 6952 Acked Read 10 Sent Done Done false
23079 6953 Acked Read 1 Sent Done Done false
23080 6954 Acked Read 1 Sent Done Done false
23081 6955 Acked Flush 0 Sent Done Done false
23082 6956 Acked Write 3 Sent Done Done false
23083 6957 Acked Flush 0 Sent Done Done false
23084 6958 Acked Read 6 Sent Done Done false
23085 6959 Acked Write 1 Sent Done Done false
23086 6960 Acked Write 6 Sent Done Done false
23087 6961 Acked Read 4 Sent Done Done false
23088 6962 Acked Read 4 Sent Done Done false
23089 6963 Acked Flush 0 Sent Done Done false
23090 6964 Acked Flush 0 Sent Done Done false
23091 6965 Acked Read 3 Sent Done Done false
23092 6966 Acked Write 7 Sent Done Done false
23093 6967 Acked Write 5 Sent Done Done false
23094 6968 Acked Write 3 Sent Done Done false
23095 6969 Acked Write 5 Sent Done Done false
23096 6970 Acked Read 9 Sent Done Done false
23097 6971 Acked Read 3 Sent Done Done false
23098 6972 Acked Flush 0 Sent Done Done false
23099 6973 Acked Read 4 Sent Done Done false
23100 6974 Acked Read 3 Sent Done Done false
23101 6975 Acked Flush 0 Sent Done Done false
23102 6976 Acked Read 10 Sent Done Done false
23103 6977 Acked Write 10 Sent Done Done false
23104 6978 Acked Read 10 Sent Done Done false
23105 6979 Acked Read 1 Sent Done Done false
23106 6980 Acked Flush 0 Sent Done Done false
23107 6981 Acked Write 4 Sent Done Done false
23108 6982 Acked Read 7 Sent Done Done false
23109 6983 Acked Write 3 Sent Done Done false
23110 6984 Acked Read 1 Sent Done Done false
23111 6985 Acked Write 7 Sent Done Done false
23112 6986 Acked Write 3 Sent Done Done false
23113 6987 Acked Write 7 Sent Done Done false
23114 6988 Acked Write 9 Sent Done Done false
23115 6989 Acked Write 7 Sent Done Done false
23116 6990 Acked Read 3 Sent Done Done false
23117 6991 Acked Write 8 Sent Done Done false
23118 6992 Acked Write 7 Sent Done Done false
23119 6993 Acked Write 4 Sent Done Done false
23120 6994 Acked Read 5 Sent Done Done false
23121 6995 Acked Read 4 Sent Done Done false
23122 6996 Acked Flush 0 Sent Done Done false
23123 6997 Acked Flush 0 Sent Done Done false
23124 6998 Acked Read 1 Sent Done Done false
23125 6999 Acked Write 3 Sent Done Done false
23126 7000 Acked Flush 0 Sent Done Done false
23127 7001 Acked Write 1 Sent Done Done false
23128 7002 Acked Write 4 Sent Done Done false
23129 7003 Acked Write 8 Sent Done Done false
23130 7004 Acked Write 5 Sent Done Done false
23131 7005 Acked Write 5 Sent Done Done false
23132 7006 Acked Read 10 Sent Done Done false
23133 7007 Acked Flush 0 Sent Done Done false
23134 7008 Acked Read 5 Sent Done Done false
23135 7009 Acked Write 6 Sent Done Done false
23136 7010 Acked Write 6 Sent Done Done false
23137 7011 Acked Read 3 Sent Done Done false
23138 7012 Acked Write 10 Sent Done Done false
23139 7013 Acked Write 7 Sent Done Done false
23140 7014 Acked Write 9 Sent Done Done false
23141 7015 Acked Read 3 Sent Done Done false
23142 7016 Acked Write 7 Sent Done Done false
23143 7017 Acked Read 3 Sent Done Done false
23144 7018 Acked Read 9 Sent Done Done false
23145 7019 Acked Write 1 Sent Done Done false
23146 7020 Acked Read 9 Sent Done Done false
23147 7021 Acked Write 8 Sent Done Done false
23148 7022 Acked Write 10 Sent Done Done false
23149 7023 Acked Read 2 Sent Done Done false
23150 7024 Acked Write 5 Sent Done Done false
23151 7025 Acked Read 8 Sent Done Done false
23152 7026 Acked Write 9 Sent Done Done false
23153 7027 Acked Read 3 Sent Done Done false
23154 7028 Acked Read 1 Sent Done Done false
23155 7029 Acked Read 9 Sent Done Done false
23156 7030 Acked Read 6 Sent Done Done false
23157 7031 Acked Read 8 Sent Done Done false
23158 7032 Acked Write 4 Sent Done Done false
23159 7033 Acked Write 10 Sent Done Done false
23160 7034 Acked Flush 0 Sent Done Done false
23161 7035 Acked Flush 0 Sent Done Done false
23162 7036 Acked Read 9 Sent Done Done false
23163 7037 Acked Write 3 Sent Done Done false
23164 7038 Acked Read 2 Sent Done Done false
23165 7039 Acked Write 3 Sent Done Done false
23166 7040 Acked Read 1 Sent Done Done false
23167 7041 Acked Read 9 Sent Done Done false
23168 7042 Acked Write 8 Sent Done Done false
23169 7043 Acked Write 4 Sent Done Done false
23170 7044 Acked Read 10 Sent Done Done false
23171 7045 Acked Read 2 Sent Done Done false
23172 STATES DS:0 DS:1 DS:2 TOTAL
23173 Sent 301 0 0 301
23174 Done 0 301 301 602
23175 Skipped 0 0 0 0
23176 Error 0 0 0 0
23177 Last Flush: 6744 7035 7035
23178 Downstairs last five completed: 6744 6743 6742 6741 6740
23179 Upstairs last five completed: 7045 7044 7043 7042 7041
23180 CLIENT: Up:0 ds:301 act:2
23181 [8810] got start action
23182 [8810] I am going to restart
23183 Starting downstairs at port 8810
23184 Make output file at "/tmp/dsc/downstairs-8810.txt"
23185 [8810][0] reports Starting
23186 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1219)
23187 [8810][0] reports Running
23188 ----------------------------------------------------------------
23189 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23190 States: Offline Active Active
23191 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23192 6745 Acked Write 5 Sent Done Done false
23193 6746 Acked Read 5 Sent Done Done false
23194 6747 Acked Read 4 Sent Done Done false
23195 6748 Acked Write 9 Sent Done Done false
23196 6749 Acked Read 4 Sent Done Done false
23197 6750 Acked Read 5 Sent Done Done false
23198 6751 Acked Write 1 Sent Done Done false
23199 6752 Acked Write 6 Sent Done Done false
23200 6753 Acked Read 4 Sent Done Done false
23201 6754 Acked Read 5 Sent Done Done false
23202 6755 Acked Write 3 Sent Done Done false
23203 6756 Acked Write 5 Sent Done Done false
23204 6757 Acked Write 5 Sent Done Done false
23205 6758 Acked Write 2 Sent Done Done false
23206 6759 Acked Read 9 Sent Done Done false
23207 6760 Acked Read 10 Sent Done Done false
23208 6761 Acked Read 3 Sent Done Done false
23209 6762 Acked Write 5 Sent Done Done false
23210 6763 Acked Write 2 Sent Done Done false
23211 6764 Acked Read 8 Sent Done Done false
23212 6765 Acked Write 6 Sent Done Done false
23213 6766 Acked Write 8 Sent Done Done false
23214 6767 Acked Read 6 Sent Done Done false
23215 6768 Acked Read 2 Sent Done Done false
23216 6769 Acked Write 10 Sent Done Done false
23217 6770 Acked Write 9 Sent Done Done false
23218 6771 Acked Read 3 Sent Done Done false
23219 6772 Acked Write 7 Sent Done Done false
23220 6773 Acked Write 9 Sent Done Done false
23221 6774 Acked Read 9 Sent Done Done false
23222 6775 Acked Read 6 Sent Done Done false
23223 6776 Acked Write 4 Sent Done Done false
23224 6777 Acked Read 8 Sent Done Done false
23225 6778 Acked Read 2 Sent Done Done false
23226 6779 Acked Read 1 Sent Done Done false
23227 6780 Acked Read 4 Sent Done Done false
23228 6781 Acked Write 8 Sent Done Done false
23229 6782 Acked Read 1 Sent Done Done false
23230 6783 Acked Write 10 Sent Done Done false
23231 6784 Acked Write 6 Sent Done Done false
23232 6785 Acked Read 6 Sent Done Done false
23233 6786 Acked Read 4 Sent Done Done false
23234 6787 Acked Write 4 Sent Done Done false
23235 6788 Acked Read 7 Sent Done Done false
23236 6789 Acked Write 3 Sent Done Done false
23237 6790 Acked Read 9 Sent Done Done false
23238 6791 Acked Read 1 Sent Done Done false
23239 6792 Acked Write 3 Sent Done Done false
23240 6793 Acked Write 3 Sent Done Done false
23241 6794 Acked Write 8 Sent Done Done false
23242 6795 Acked Write 5 Sent Done Done false
23243 6796 Acked Flush 0 Sent Done Done false
23244 6797 Acked Write 1 Sent Done Done false
23245 6798 Acked Read 8 Sent Done Done false
23246 6799 Acked Read 5 Sent Done Done false
23247 6800 Acked Write 6 Sent Done Done false
23248 6801 Acked Write 6 Sent Done Done false
23249 6802 Acked Write 8 Sent Done Done false
23250 6803 Acked Read 8 Sent Done Done false
23251 6804 Acked Write 4 Sent Done Done false
23252 6805 Acked Read 3 Sent Done Done false
23253 6806 Acked Write 4 Sent Done Done false
23254 6807 Acked Write 4 Sent Done Done false
23255 6808 Acked Write 5 Sent Done Done false
23256 6809 Acked Read 5 Sent Done Done false
23257 6810 Acked Write 2 Sent Done Done false
23258 6811 Acked Read 6 Sent Done Done false
23259 6812 Acked Flush 0 Sent Done Done false
23260 6813 Acked Write 4 Sent Done Done false
23261 6814 Acked Write 6 Sent Done Done false
23262 6815 Acked Read 2 Sent Done Done false
23263 6816 Acked Write 10 Sent Done Done false
23264 6817 Acked Read 2 Sent Done Done false
23265 6818 Acked Read 6 Sent Done Done false
23266 6819 Acked Flush 0 Sent Done Done false
23267 6820 Acked Write 4 Sent Done Done false
23268 6821 Acked Flush 0 Sent Done Done false
23269 6822 Acked Read 5 Sent Done Done false
23270 6823 Acked Read 9 Sent Done Done false
23271 6824 Acked Write 9 Sent Done Done false
23272 6825 Acked Read 9 Sent Done Done false
23273 6826 Acked Flush 0 Sent Done Done false
23274 6827 Acked Write 10 Sent Done Done false
23275 6828 Acked Read 8 Sent Done Done false
23276 6829 Acked Write 9 Sent Done Done false
23277 6830 Acked Write 3 Sent Done Done false
23278 6831 Acked Read 8 Sent Done Done false
23279 6832 Acked Write 5 Sent Done Done false
23280 6833 Acked Read 1 Sent Done Done false
23281 6834 Acked Read 4 Sent Done Done false
23282 6835 Acked Read 2 Sent Done Done false
23283 6836 Acked Read 7 Sent Done Done false
23284 6837 Acked Read 10 Sent Done Done false
23285 6838 Acked Read 2 Sent Done Done false
23286 6839 Acked Flush 0 Sent Done Done false
23287 6840 Acked Read 5 Sent Done Done false
23288 6841 Acked Read 7 Sent Done Done false
23289 6842 Acked Write 2 Sent Done Done false
23290 6843 Acked Flush 0 Sent Done Done false
23291 6844 Acked Write 7 Sent Done Done false
23292 6845 Acked Read 6 Sent Done Done false
23293 6846 Acked Flush 0 Sent Done Done false
23294 6847 Acked Read 3 Sent Done Done false
23295 6848 Acked Flush 0 Sent Done Done false
23296 6849 Acked Read 5 Sent Done Done false
23297 6850 Acked Flush 0 Sent Done Done false
23298 6851 Acked Read 3 Sent Done Done false
23299 6852 Acked Read 4 Sent Done Done false
23300 6853 Acked Write 9 Sent Done Done false
23301 6854 Acked Flush 0 Sent Done Done false
23302 6855 Acked Read 4 Sent Done Done false
23303 6856 Acked Write 1 Sent Done Done false
23304 6857 Acked Read 4 Sent Done Done false
23305 6858 Acked Write 8 Sent Done Done false
23306 6859 Acked Write 2 Sent Done Done false
23307 6860 Acked Read 7 Sent Done Done false
23308 6861 Acked Read 3 Sent Done Done false
23309 6862 Acked Write 9 Sent Done Done false
23310 6863 Acked Write 6 Sent Done Done false
23311 6864 Acked Write 3 Sent Done Done false
23312 6865 Acked Read 7 Sent Done Done false
23313 6866 Acked Write 7 Sent Done Done false
23314 6867 Acked Write 6 Sent Done Done false
23315 6868 Acked Write 1 Sent Done Done false
23316 6869 Acked Read 4 Sent Done Done false
23317 6870 Acked Read 10 Sent Done Done false
23318 6871 Acked Write 8 Sent Done Done false
23319 6872 Acked Read 2 Sent Done Done false
23320 6873 Acked Read 7 Sent Done Done false
23321 6874 Acked Read 5 Sent Done Done false
23322 6875 Acked Read 5 Sent Done Done false
23323 6876 Acked Read 5 Sent Done Done false
23324 6877 Acked Read 1 Sent Done Done false
23325 6878 Acked Read 8 Sent Done Done false
23326 6879 Acked Read 2 Sent Done Done false
23327 6880 Acked Flush 0 Sent Done Done false
23328 6881 Acked Read 3 Sent Done Done false
23329 6882 Acked Read 2 Sent Done Done false
23330 6883 Acked Write 1 Sent Done Done false
23331 6884 Acked Read 2 Sent Done Done false
23332 6885 Acked Read 10 Sent Done Done false
23333 6886 Acked Write 10 Sent Done Done false
23334 6887 Acked Read 3 Sent Done Done false
23335 6888 Acked Read 8 Sent Done Done false
23336 6889 Acked Write 2 Sent Done Done false
23337 6890 Acked Write 3 Sent Done Done false
23338 6891 Acked Write 6 Sent Done Done false
23339 6892 Acked Flush 0 Sent Done Done false
23340 6893 Acked Read 3 Sent Done Done false
23341 6894 Acked Read 1 Sent Done Done false
23342 6895 Acked Read 2 Sent Done Done false
23343 6896 Acked Read 3 Sent Done Done false
23344 6897 Acked Read 10 Sent Done Done false
23345 6898 Acked Read 8 Sent Done Done false
23346 6899 Acked Write 2 Sent Done Done false
23347 6900 Acked Write 1 Sent Done Done false
23348 6901 Acked Read 5 Sent Done Done false
23349 6902 Acked Write 8 Sent Done Done false
23350 6903 Acked Read 3 Sent Done Done false
23351 6904 Acked Write 7 Sent Done Done false
23352 6905 Acked Read 6 Sent Done Done false
23353 6906 Acked Read 8 Sent Done Done false
23354 6907 Acked Flush 0 Sent Done Done false
23355 6908 Acked Read 5 Sent Done Done false
23356 6909 Acked Write 8 Sent Done Done false
23357 6910 Acked Read 9 Sent Done Done false
23358 6911 Acked Read 10 Sent Done Done false
23359 6912 Acked Flush 0 Sent Done Done false
23360 6913 Acked Write 9 Sent Done Done false
23361 6914 Acked Read 4 Sent Done Done false
23362 6915 Acked Flush 0 Sent Done Done false
23363 6916 Acked Write 2 Sent Done Done false
23364 6917 Acked Write 9 Sent Done Done false
23365 6918 Acked Read 5 Sent Done Done false
23366 6919 Acked Write 2 Sent Done Done false
23367 6920 Acked Flush 0 Sent Done Done false
23368 6921 Acked Read 2 Sent Done Done false
23369 6922 Acked Read 8 Sent Done Done false
23370 6923 Acked Read 6 Sent Done Done false
23371 6924 Acked Flush 0 Sent Done Done false
23372 6925 Acked Read 6 Sent Done Done false
23373 6926 Acked Write 8 Sent Done Done false
23374 6927 Acked Read 10 Sent Done Done false
23375 6928 Acked Write 4 Sent Done Done false
23376 6929 Acked Write 6 Sent Done Done false
23377 6930 Acked Write 5 Sent Done Done false
23378 6931 Acked Read 4 Sent Done Done false
23379 6932 Acked Write 2 Sent Done Done false
23380 6933 Acked Write 6 Sent Done Done false
23381 6934 Acked Write 9 Sent Done Done false
23382 6935 Acked Write 3 Sent Done Done false
23383 6936 Acked Read 9 Sent Done Done false
23384 6937 Acked Read 8 Sent Done Done false
23385 6938 Acked Write 9 Sent Done Done false
23386 6939 Acked Read 3 Sent Done Done false
23387 6940 Acked Write 4 Sent Done Done false
23388 6941 Acked Read 4 Sent Done Done false
23389 6942 Acked Read 5 Sent Done Done false
23390 6943 Acked Write 4 Sent Done Done false
23391 6944 Acked Read 10 Sent Done Done false
23392 6945 Acked Write 7 Sent Done Done false
23393 6946 Acked Write 9 Sent Done Done false
23394 6947 Acked Read 7 Sent Done Done false
23395 6948 Acked Write 3 Sent Done Done false
23396 6949 Acked Write 10 Sent Done Done false
23397 6950 Acked Write 8 Sent Done Done false
23398 6951 Acked Write 3 Sent Done Done false
23399 6952 Acked Read 10 Sent Done Done false
23400 6953 Acked Read 1 Sent Done Done false
23401 6954 Acked Read 1 Sent Done Done false
23402 6955 Acked Flush 0 Sent Done Done false
23403 6956 Acked Write 3 Sent Done Done false
23404 6957 Acked Flush 0 Sent Done Done false
23405 6958 Acked Read 6 Sent Done Done false
23406 6959 Acked Write 1 Sent Done Done false
23407 6960 Acked Write 6 Sent Done Done false
23408 6961 Acked Read 4 Sent Done Done false
23409 6962 Acked Read 4 Sent Done Done false
23410 6963 Acked Flush 0 Sent Done Done false
23411 6964 Acked Flush 0 Sent Done Done false
23412 6965 Acked Read 3 Sent Done Done false
23413 6966 Acked Write 7 Sent Done Done false
23414 6967 Acked Write 5 Sent Done Done false
23415 6968 Acked Write 3 Sent Done Done false
23416 6969 Acked Write 5 Sent Done Done false
23417 6970 Acked Read 9 Sent Done Done false
23418 6971 Acked Read 3 Sent Done Done false
23419 6972 Acked Flush 0 Sent Done Done false
23420 6973 Acked Read 4 Sent Done Done false
23421 6974 Acked Read 3 Sent Done Done false
23422 6975 Acked Flush 0 Sent Done Done false
23423 6976 Acked Read 10 Sent Done Done false
23424 6977 Acked Write 10 Sent Done Done false
23425 6978 Acked Read 10 Sent Done Done false
23426 6979 Acked Read 1 Sent Done Done false
23427 6980 Acked Flush 0 Sent Done Done false
23428 6981 Acked Write 4 Sent Done Done false
23429 6982 Acked Read 7 Sent Done Done false
23430 6983 Acked Write 3 Sent Done Done false
23431 6984 Acked Read 1 Sent Done Done false
23432 6985 Acked Write 7 Sent Done Done false
23433 6986 Acked Write 3 Sent Done Done false
23434 6987 Acked Write 7 Sent Done Done false
23435 6988 Acked Write 9 Sent Done Done false
23436 6989 Acked Write 7 Sent Done Done false
23437 6990 Acked Read 3 Sent Done Done false
23438 6991 Acked Write 8 Sent Done Done false
23439 6992 Acked Write 7 Sent Done Done false
23440 6993 Acked Write 4 Sent Done Done false
23441 6994 Acked Read 5 Sent Done Done false
23442 6995 Acked Read 4 Sent Done Done false
23443 6996 Acked Flush 0 Sent Done Done false
23444 6997 Acked Flush 0 Sent Done Done false
23445 6998 Acked Read 1 Sent Done Done false
23446 6999 Acked Write 3 Sent Done Done false
23447 7000 Acked Flush 0 Sent Done Done false
23448 7001 Acked Write 1 Sent Done Done false
23449 7002 Acked Write 4 Sent Done Done false
23450 7003 Acked Write 8 Sent Done Done false
23451 7004 Acked Write 5 Sent Done Done false
23452 7005 Acked Write 5 Sent Done Done false
23453 7006 Acked Read 10 Sent Done Done false
23454 7007 Acked Flush 0 Sent Done Done false
23455 7008 Acked Read 5 Sent Done Done false
23456 7009 Acked Write 6 Sent Done Done false
23457 7010 Acked Write 6 Sent Done Done false
23458 7011 Acked Read 3 Sent Done Done false
23459 7012 Acked Write 10 Sent Done Done false
23460 7013 Acked Write 7 Sent Done Done false
23461 7014 Acked Write 9 Sent Done Done false
23462 7015 Acked Read 3 Sent Done Done false
23463 7016 Acked Write 7 Sent Done Done false
23464 7017 Acked Read 3 Sent Done Done false
23465 7018 Acked Read 9 Sent Done Done false
23466 7019 Acked Write 1 Sent Done Done false
23467 7020 Acked Read 9 Sent Done Done false
23468 7021 Acked Write 8 Sent Done Done false
23469 7022 Acked Write 10 Sent Done Done false
23470 7023 Acked Read 2 Sent Done Done false
23471 7024 Acked Write 5 Sent Done Done false
23472 7025 Acked Read 8 Sent Done Done false
23473 7026 Acked Write 9 Sent Done Done false
23474 7027 Acked Read 3 Sent Done Done false
23475 7028 Acked Read 1 Sent Done Done false
23476 7029 Acked Read 9 Sent Done Done false
23477 7030 Acked Read 6 Sent Done Done false
23478 7031 Acked Read 8 Sent Done Done false
23479 7032 Acked Write 4 Sent Done Done false
23480 7033 Acked Write 10 Sent Done Done false
23481 7034 Acked Flush 0 Sent Done Done false
23482 7035 Acked Flush 0 Sent Done Done false
23483 7036 Acked Read 9 Sent Done Done false
23484 7037 Acked Write 3 Sent Done Done false
23485 7038 Acked Read 2 Sent Done Done false
23486 7039 Acked Write 3 Sent Done Done false
23487 7040 Acked Read 1 Sent Done Done false
23488 7041 Acked Read 9 Sent Done Done false
23489 7042 Acked Write 8 Sent Done Done false
23490 7043 Acked Write 4 Sent Done Done false
23491 7044 Acked Read 10 Sent Done Done false
23492 7045 Acked Read 2 Sent Done Done false
23493 7046 Acked Flush 0 Sent Done Done false
23494 STATES DS:0 DS:1 DS:2 TOTAL
23495 Sent 302 0 0 302
23496 Done 0 302 302 604
23497 Skipped 0 0 0 0
23498 Error 0 0 0 0
23499 Last Flush: 6744 7046 7046
23500 Downstairs last five completed: 6744 6743 6742 6741 6740
23501 Upstairs last five completed: 7046 7045 7044 7043 7042
23502 CLIENT: Up:0 ds:302 act:2
235032024-11-07T14:08:13.447ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235042024-11-07T14:08:13.447ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41929) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235052024-11-07T14:08:13.448ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235062024-11-07T14:08:13.449ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235072024-11-07T14:08:13.449ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235082024-11-07T14:08:13.449ZINFOcrucible: send last flush ID to this DS: 6744 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235092024-11-07T14:08:13.449ZINFOcrucible: Replied this last flush ID: 6744 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235102024-11-07T14:08:13.449ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235112024-11-07T14:08:13.449ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235122024-11-07T14:08:13.449ZINFOcrucible: [0] client re-new 302 jobs since flush 6744 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235132024-11-07T14:08:13.449ZINFOcrucible: [0] Replayed 302 jobs since flush: 6744 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
23514 ----------------------------------------------------------------
23515 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23516 Downstairs last five completed: 7046 7045 7044 7043 7042
23517 Upstairs last five completed: 7046 7045 7044 7043 7042
23518 CLIENT: Up:0 ds:0 act:3
23519 Replay: All jobs finished
23520 Main task has work to do, go find it
23521 got dsc Stop(2)
23522 stop 2
23523 Nov 07 14:08:15.731 INFO request completed[8830] Got stop action so:false kr:false
23524 , latency_us: 327, response_code: 204, uri: /stop/cid/2, method: POST, req_id: ae66e83b-9992-466d-8b2a-00e3f8e2d466, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
235252024-11-07T14:08:15.732ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235262024-11-07T14:08:15.732ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235272024-11-07T14:08:15.732ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235282024-11-07T14:08:15.732ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
235292024-11-07T14:08:15.732ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
23530 Nov 07 14:08:15.732 INFO request completed, latency_us: 166, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8db2a8a5-fcc1-4f00-aa2d-ed7c6992ff49, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
235312024-11-07T14:08:15.732ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
23532 [8830] Exited with: ExitStatus(unix_wait_status(9))
23533 [8830][2] reports Exit
23534 Nov 07 14:08:19.734 INFO request completed, latency_us: 187, response_code: 200, uri: /state/cid/2, method: GET, req_id: e5945e1d-fb57-4a4c-9664-a28447d9014c, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
23535 001/300 Write block 2257 len 12288 data: 3 3 3
23536 002/300 Read block 2450 len 40960
23537 003/300 Read block 550 len 16384
23538 004/300 Read block 2566 len 8192
23539 005/300 Read block 1584 len 8192
23540 006/300 Read block 3892 len 16384
23541 007/300 Write block 4619 len 28672 data: 4 5 3 3 2 3 3
23542 008/300 Write block 513 len 16384 data: 4 5 6 5
23543 009/300 Read block 1790 len 8192
23544 010/300 Write block 4266 len 8192 data: 5 3
23545 011/300 Read block 4043 len 24576
23546 012/300 Read block 2439 len 4096
23547 013/300 Write block 4304 len 40960 data: 3 3 5 6 5 7 6 8 7 7
23548 014/300 Write block 839 len 12288 data: 2 2 2
23549 015/300 Read block 283 len 12288
23550 016/300 Read block 729 len 12288
23551 017/300 Read block 4116 len 4096
23552 018/300 Read block 2725 len 24576
23553 019/300 Read block 1561 len 32768
23554 020/300 Read block 3814 len 28672
23555 021/300 Write block 4590 len 28672 data: 4 4 4 4 4 4 4
23556 022/300 Write block 1953 len 36864 data: 4 4 4 4 3 4 5 6 6
23557 023/300 Write block 1010 len 20480 data: 5 4 5 5 6
23558 024/300 Flush
23559 025/300 Read block 4903 len 12288
23560 026/300 Write block 2066 len 16384 data: 2 2 3 3
23561 027/300 Read block 1227 len 28672
23562 028/300 Read block 238 len 36864
23563 029/300 Write block 467 len 20480 data: 7 6 6 6 4
23564 030/300 Write block 3548 len 40960 data: 8 7 6 7 9 8 6 7 6 3
23565 031/300 Read block 2273 len 24576
23566 032/300 Read block 880 len 16384
23567 033/300 Read block 4807 len 20480
23568 034/300 Read block 1790 len 28672
23569 035/300 Write block 3475 len 40960 data: 3 3 6 7 6 6 5 5 6 6
23570 036/300 Read block 4858 len 28672
23571 037/300 Read block 1832 len 36864
23572 038/300 Read block 2713 len 4096
23573 039/300 Write block 3119 len 4096 data: 5
23574 040/300 Write block 706 len 16384 data: 7 9 8 9
23575 041/300 Read block 4663 len 24576
23576 042/300 Write block 1309 len 40960 data: 4 4 4 4 4 5 4 4 6 5
23577 043/300 Read block 1869 len 28672
23578 044/300 Read block 1880 len 20480
23579 045/300 Write block 2137 len 20480 data: 2 3 4 3 3
23580 046/300 Write block 656 len 16384 data: 5 4 4 4
23581 047/300 Write block 2677 len 8192 data: 4 2
23582 048/300 Write block 1005 len 24576 data: 3 3 2 4 4 6
23583 049/300 Write block 3808 len 32768 data: 3 2 2 2 2 4 5 6
23584 050/300 Read block 4391 len 32768
23585 051/300 Write block 2368 len 20480 data: 4 4 4 4 3
23586 052/300 Read block 118 len 12288
23587 053/300 Read block 4780 len 24576
23588 054/300 Read block 1268 len 40960
23589 055/300 Read block 2723 len 16384
23590 056/300 Flush
23591 057/300 Read block 2685 len 8192
23592 058/300 Read block 3976 len 4096
23593 059/300 Flush
23594 060/300 Flush
23595 061/300 Write block 3590 len 12288 data: 6 5 3
23596 062/300 Flush
23597 063/300 Read block 4537 len 8192
23598 064/300 Read block 1395 len 40960
23599 065/300 Write block 3711 len 40960 data: 7 6 6 6 7 8 9 10 10 10
23600 066/300 Read block 84 len 20480
23601 067/300 Write block 233 len 24576 data: 4 5 4 4 5 5
23602 068/300 Flush
23603 069/300 Read block 286 len 40960
23604 070/300 Read block 3855 len 16384
23605 071/300 Read block 4632 len 16384
23606 072/300 Write block 3502 len 32768 data: 2 3 3 3 4 5 5 6
23607 073/300 Write block 3501 len 40960 data: 3 3 4 4 4 5 6 6 7 6
23608 074/300 Write block 2898 len 16384 data: 3 3 3 4
23609 075/300 Read block 148 len 36864
23610 076/300 Read block 2114 len 20480
23611 077/300 Write block 3971 len 12288 data: 6 8 6
23612 078/300 Read block 2586 len 20480
23613 079/300 Write block 2123 len 4096 data: 2
23614 080/300 Write block 3649 len 36864 data: 3 4 4 3 3 4 3 3 4
23615 081/300 Read block 1121 len 12288
23616 082/300 Read block 1398 len 16384
23617 083/300 Read block 2139 len 16384
23618 084/300 Write block 2092 len 12288 data: 3 2 2
23619 085/300 Write block 4314 len 4096 data: 8
23620 086/300 Write block 2418 len 12288 data: 6 8 8
23621 087/300 Read block 1844 len 28672
23622 088/300 Read block 63 len 36864
23623 089/300 Read block 1984 len 12288
23624 090/300 Flush
23625 091/300 Read block 106 len 28672
23626 092/300 Write block 70 len 20480 data: 3 3 6 5 4
23627 093/300 Write block 2958 len 40960 data: 4 4 4 3 4 4 5 4 3 3
23628 094/300 Write block 3672 len 36864 data: 3 3 5 5 4 5 4 4 4
23629 095/300 Write block 1170 len 28672 data: 3 5 6 6 6 5 5
23630 096/300 Read block 306 len 12288
23631 097/300 Write block 1316 len 28672 data: 5 7 6 5 5 5 5
23632 098/300 Read block 2569 len 24576
23633 099/300 Write block 1509 len 40960 data: 8 5 5 5 4 3 3 3 3 3
23634 100/300 Read block 2540 len 24576
23635 101/300 Read block 4253 len 36864
23636 102/300 Read block 4920 len 12288
23637 103/300 Read block 1784 len 40960
23638 104/300 Read block 763 len 4096
23639 105/300 Write block 3699 len 20480 data: 9 7 6 7 8
23640 106/300 Read block 3757 len 28672
23641 107/300 Read block 128 len 12288
23642 108/300 Write block 769 len 36864 data: 7 5 6 6 6 5 5 5 3
23643 109/300 Write block 2361 len 40960 data: 5 5 5 5 7 5 4 5 5 5
23644 110/300 Read block 4568 len 4096
23645 111/300 Read block 1643 len 16384
23646 112/300 Flush
23647 113/300 Flush
23648 114/300 Read block 3687 len 20480
23649 115/300 Write block 3459 len 20480 data: 3 3 3 5 5
23650 116/300 Read block 1554 len 24576
23651 117/300 Read block 3513 len 16384
23652 118/300 Flush
23653 119/300 Read block 3144 len 24576
23654 120/300 Read block 343 len 16384
23655 121/300 Read block 1081 len 20480
23656 122/300 Write block 653 len 32768 data: 5 5 5 6 5 5 5 4
23657 123/300 Read block 4718 len 12288
23658 124/300 Read block 2578 len 40960
23659 125/300 Read block 494 len 36864
23660 126/300 Read block 3505 len 24576
23661 127/300 Write block 808 len 24576 data: 3 3 4 4 2 2
23662 128/300 Write block 1232 len 36864 data: 4 4 5 5 4 4 4 4 4
23663 129/300 Read block 3721 len 28672
23664 130/300 Write block 223 len 4096 data: 3
23665 131/300 Read block 1639 len 24576
23666 132/300 Write block 4512 len 12288 data: 5 5 5
23667 133/300 Read block 739 len 8192
23668 134/300 Write block 3734 len 24576 data: 4 5 5 5 7 8
23669 135/300 Read block 105 len 20480
23670 136/300 Read block 2046 len 8192
23671 137/300 Read block 1070 len 8192
23672 138/300 Read block 1100 len 16384
23673 139/300 Read block 3400 len 20480
23674 140/300 Read block 1088 len 24576
23675 141/300 Write block 3556 len 32768 data: 7 4 5 5 4 5 4 4
23676 142/300 Read block 1404 len 16384
23677 143/300 Read block 4736 len 36864
23678 144/300 Flush
23679 145/300 Read block 2121 len 24576
23680 146/300 Read block 3961 len 36864
23681 147/300 Flush
23682 148/300 Read block 1009 len 4096
23683 149/300 Read block 4124 len 32768
23684 150/300 Flush
23685 151/300 Read block 983 len 36864
23686 152/300 Read block 4877 len 20480
23687 153/300 Read block 56 len 16384
23688 154/300 Read block 973 len 20480
23689 155/300 Write block 410 len 32768 data: 7 7 4 3 3 5 5 5
23690 156/300 Read block 3042 len 40960
23691 157/300 Write block 4783 len 28672 data: 2 2 3 3 4 6 6
23692 158/300 Read block 25 len 32768
23693 159/300 Write block 3083 len 12288 data: 8 8 7
23694 160/300 Write block 1131 len 4096 data: 4
23695 161/300 Read block 197 len 28672
23696 162/300 Write block 4658 len 16384 data: 4 4 4 5
23697 163/300 Read block 3028 len 40960
23698 164/300 Write block 3791 len 12288 data: 6 6 6
23699 165/300 Write block 344 len 16384 data: 6 7 5 5
23700 166/300 Read block 682 len 12288
23701 167/300 Write block 3730 len 32768 data: 3 5 5 4 5 6 6 6
23702 168/300 Read block 3069 len 36864
23703 169/300 Write block 1828 len 4096 data: 6
23704 170/300 Flush
23705 171/300 Write block 675 len 8192 data: 3 2
23706 172/300 Write block 675 len 28672 data: 4 3 3 5 5 5 4
23707 173/300 Write block 263 len 20480 data: 6 4 5 5 4
23708 174/300 Write block 1868 len 24576 data: 4 3 3 3 3 4
23709 175/300 Read block 846 len 28672
23710 176/300 Flush
23711 177/300 Flush
23712 178/300 Write block 4095 len 12288 data: 3 3 3
23713 179/300 Write block 732 len 12288 data: 6 6 5
23714 180/300 Write block 4285 len 8192 data: 3 4
23715 181/300 Write block 4207 len 40960 data: 4 2 2 3 4 5 5 4 4 4
23716 182/300 Read block 4718 len 40960
23717 183/300 Write block 2429 len 4096 data: 5
23718 184/300 Write block 2664 len 8192 data: 8 8
23719 185/300 Read block 4739 len 40960
23720 186/300 Write block 4271 len 40960 data: 8 8 7 6 6 5 6 5 4 5
23721 187/300 Read block 1845 len 36864
23722 188/300 Write block 1965 len 36864 data: 4 5 6 5 5 5 4 5 5
23723 189/300 Read block 4586 len 4096
23724 190/300 Read block 4275 len 8192
23725 191/300 Write block 1898 len 16384 data: 3 3 3 4
23726 192/300 Write block 2149 len 8192 data: 5 7
23727 193/300 Write block 4481 len 28672 data: 7 8 7 6 7 6 6
23728 194/300 Write block 2900 len 16384 data: 4 5 4 4
23729 195/300 Write block 8 len 20480 data: 4 3 4 4 5
23730 196/300 Read block 3280 len 8192
23731 197/300 Read block 3476 len 32768
23732 198/300 Write block 1311 len 32768 data: 5 5 5 6 5 6 8 7
23733 199/300 Read block 2035 len 32768
23734 200/300 Write block 2641 len 16384 data: 6 7 7 5
23735 201/300 Write block 2630 len 16384 data: 2 4 4 4
23736 202/300 Write block 1397 len 16384 data: 5 5 5 5
23737 203/300 Write block 2666 len 12288 data: 6 5 5
23738 204/300 Read block 4554 len 4096
23739 205/300 Read block 3538 len 32768
23740 206/300 Read block 828 len 16384
23741 207/300 Read block 3667 len 20480
23742 208/300 Write block 4671 len 4096 data: 5
23743 209/300 Read block 1837 len 28672
23744 210/300 Read block 3893 len 36864
23745 211/300 Read block 3188 len 32768
23746 212/300 Read block 417 len 24576
23747 213/300 Read block 979 len 40960
23748 214/300 Flush
23749 215/300 Write block 982 len 32768 data: 5 6 6 5 5 5 5 5
23750 216/300 Read block 3692 len 4096
23751 217/300 Read block 1573 len 8192
23752 218/300 Read block 4173 len 40960
23753 219/300 Write block 3278 len 24576 data: 5 6 6 5 6 7
23754 220/300 Read block 221 len 4096
23755 221/300 Flush
23756 222/300 Write block 1904 len 12288 data: 4 3 3
23757 223/300 Read block 2846 len 40960
23758 224/300 Read block 557 len 36864
23759 225/300 Read block 3477 len 40960
23760 226/300 Read block 4428 len 24576
23761 227/300 Write block 158 len 16384 data: 5 7 8 8
23762 228/300 Write block 72 len 32768 data: 7 6 5 4 4 4 4 5
23763 229/300 Read block 694 len 12288
23764 230/300 Flush
23765 231/300 Flush
23766 232/300 Flush
23767 233/300 Write block 4893 len 32768 data: 5 6 7 8 6 5 5 5
23768 234/300 Write block 1596 len 16384 data: 3 5 6 7
23769 235/300 Flush
23770 236/300 Read block 4255 len 24576
23771 237/300 Read block 547 len 4096
23772 238/300 Write block 2079 len 8192 data: 6 6
23773 239/300 Flush
23774 240/300 Write block 2434 len 40960 data: 4 4 4 6 7 7 6 6 6 6
23775 241/300 Read block 321 len 12288
23776 242/300 Flush
23777 243/300 Read block 1296 len 16384
23778 244/300 Write block 506 len 20480 data: 3 3 3 3 4
23779 245/300 Read block 494 len 24576
23780 246/300 Read block 2848 len 4096
23781 247/300 Write block 4340 len 12288 data: 5 4 4
23782 248/300 Read block 4322 len 40960
23783 249/300 Read block 2081 len 16384
23784 250/300 Read block 2071 len 4096
23785 251/300 Write block 1568 len 40960 data: 6 5 5 4 5 6 6 6 6 4
23786 252/300 Write block 3320 len 12288 data: 4 5 5
23787 253/300 Write block 161 len 12288 data: 9 7 7
23788 254/300 Write block 1869 len 36864 data: 4 4 4 4 5 4 5 6 7
23789 255/300 Read block 88 len 8192
23790 256/300 Write block 4888 len 4096 data: 2
23791 257/300 Read block 328 len 32768
23792 258/300 Write block 2851 len 36864 data: 5 7 8 9 6 5 5 5 5
23793 259/300 Read block 3295 len 4096
23794 260/300 Read block 899 len 4096
23795 261/300 Write block 419 len 32768 data: 4 5 4 5 5 3 3 3
23796 262/300 Write block 2433 len 4096 data: 4
23797 263/300 Write block 473 len 28672 data: 4 4 3 4 5 4 4
23798 264/300 Read block 2409 len 20480
23799 265/300 Write block 4437 len 20480 data: 6 7 8 6 7
23800 266/300 Write block 3026 len 32768 data: 5 4 7 7 6 6 6 5
23801 267/300 Read block 4036 len 12288
23802 268/300 Read block 347 len 24576
23803 269/300 Read block 2214 len 4096
23804 270/300 Write block 4367 len 12288 data: 6 6 6
23805 271/300 Write block 4594 len 12288 data: 5 5 5
23806 272/300 Read block 4642 len 20480
23807 273/300 Write block 4328 len 28672 data: 3 2 3 4 4 4 4
23808 274/300 Write block 4820 len 32768 data: 5 4 4 3 3 3 2 2
23809 275/300 Flush
23810 276/300 Write block 685 len 8192 data: 4 4
23811 277/300 Flush
23812 278/300 Write block 2826 len 4096 data: 7
23813 279/300 Read block 852 len 4096
23814 280/300 Write block 1916 len 16384 data: 5 4 4 6
23815 281/300 Write block 3351 len 24576 data: 2 4 4 4 4 4
23816 282/300 Read block 3106 len 8192
23817 283/300 Flush
23818 284/300 Read block 2014 len 28672
23819 285/300 Read block 997 len 28672
23820 286/300 Write block 3917 len 16384 data: 3 3 3 3
23821 287/300 Write block 2997 len 4096 data: 5
23822 288/300 Write block 413 len 40960 data: 4 4 6 6 6 4 5 6 5 6
23823 289/300 Write block 3227 len 20480 data: 3 3 3 4 3
23824 290/300 Write block 4825 len 24576 data: 4 3 3 2 2 2
23825 291/300 Write block 1321 len 16384 data: 6 6 3 5
23826 292/300 Write block 1855 len 4096 data: 6
23827 293/300 Read block 714 len 4096
23828 294/300 Read block 2791 len 16384
23829 295/300 Read block 2848 len 32768
23830 296/300 Read block 661 len 8192
23831 297/300 Read block 54 len 8192
23832 298/300 Read block 2525 len 4096
23833 299/300 Read block 4120 len 32768
23834 300/300 Write block 1984 len 32768 data: 7 7 8 8 7 5 3 2
23835 301/300 Write block 4364 len 36864 data: 4 4 6 7 7 7 6 7 5
23836 Main task has work to do, go find it
23837 got dsc Start(2)
23838 start 2
23839 Nov 07 14:08:20.008 INFO request completed, latency_us: 273, response_code: 204, uri: /start/cid/2, method: POST, req_id: 93689ae7-826a-4c1c-8eac-98a449781d32, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
23840 Replay: started 2, returned:Ok(())
23841 ----------------------------------------------------------------
23842 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23843 States: Active Active Offline
23844 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23845 7047 Acked Write 3 Done Done Sent false
23846 7048 Acked Read 10 Done Done Sent false
23847 7049 Acked Read 4 Done Done Sent false
23848 7050 Acked Read 2 Done Done Sent false
23849 7051 Acked Read 2 Done Done Sent false
23850 7052 Acked Read 4 Done Done Sent false
23851 7053 Acked Write 7 Done Done Sent false
23852 7054 Acked Write 4 Done Done Sent false
23853 7055 Acked Read 2 Done Done Sent false
23854 7056 Acked Write 2 Done Done Sent false
23855 7057 Acked Read 6 Done Done Sent false
23856 7058 Acked Read 1 Done Done Sent false
23857 7059 Acked Write 10 Done Done Sent false
23858 7060 Acked Write 3 Done Done Sent false
23859 7061 Acked Read 3 Done Done Sent false
23860 7062 Acked Read 3 Done Done Sent false
23861 7063 Acked Read 1 Done Done Sent false
23862 7064 Acked Read 6 Done Done Sent false
23863 7065 Acked Read 8 Done Done Sent false
23864 7066 Acked Read 7 Done Done Sent false
23865 7067 Acked Write 7 Done Done Sent false
23866 7068 Acked Write 9 Done Done Sent false
23867 7069 Acked Write 5 Done Done Sent false
23868 7070 Acked Flush 0 Done Done Sent false
23869 7071 Acked Read 3 Done Done Sent false
23870 7072 Acked Write 4 Done Done Sent false
23871 7073 Acked Read 7 Done Done Sent false
23872 7074 Acked Read 9 Done Done Sent false
23873 7075 Acked Write 5 Done Done Sent false
23874 7076 Acked Write 10 Done Done Sent false
23875 7077 Acked Read 6 Done Done Sent false
23876 7078 Acked Read 4 Done Done Sent false
23877 7079 Acked Read 5 Done Done Sent false
23878 7080 Acked Read 7 Done Done Sent false
23879 7081 Acked Write 10 Done Done Sent false
23880 7082 Acked Read 7 Done Done Sent false
23881 7083 Acked Read 9 Done Done Sent false
23882 7084 Acked Read 1 Done Done Sent false
23883 7085 Acked Write 1 Done Done Sent false
23884 7086 Acked Write 4 Done Done Sent false
23885 7087 Acked Read 6 Done Done Sent false
23886 7088 Acked Write 10 Done Done Sent false
23887 7089 Acked Read 7 Done Done Sent false
23888 7090 Acked Read 5 Done Done Sent false
23889 7091 Acked Write 5 Done Done Sent false
23890 7092 Acked Write 4 Done Done Sent false
23891 7093 Acked Write 2 Done Done Sent false
23892 7094 Acked Write 6 Done Done Sent false
23893 7095 Acked Write 8 Done Done Sent false
23894 7096 Acked Read 8 Done Done Sent false
23895 7097 Acked Write 5 Done Done Sent false
23896 7098 Acked Read 3 Done Done Sent false
23897 7099 Acked Read 6 Done Done Sent false
23898 7100 Acked Read 10 Done Done Sent false
23899 7101 Acked Read 4 Done Done Sent false
23900 7102 Acked Flush 0 Done Done Sent false
23901 7103 Acked Read 2 Done Done Sent false
23902 7104 Acked Read 1 Done Done Sent false
23903 7105 Acked Flush 0 Done Done Sent false
23904 7106 Acked Flush 0 Done Done Sent false
23905 7107 Acked Write 3 Done Done Sent false
23906 7108 Acked Flush 0 Done Done Sent false
23907 7109 Acked Read 2 Done Done Sent false
23908 7110 Acked Read 10 Done Done Sent false
23909 7111 Acked Write 10 Done Done Sent false
23910 7112 Acked Read 5 Done Done Sent false
23911 7113 Acked Write 6 Done Done Sent false
23912 7114 Acked Flush 0 Done Done Sent false
23913 7115 Acked Read 10 Done Done Sent false
23914 7116 Acked Read 4 Done Done Sent false
23915 7117 Acked Read 4 Done Done Sent false
23916 7118 Acked Write 8 Done Done Sent false
23917 7119 Acked Write 10 Done Done Sent false
23918 7120 Acked Write 4 Done Done Sent false
23919 7121 Acked Read 9 Done Done Sent false
23920 7122 Acked Read 5 Done Done Sent false
23921 7123 Acked Write 3 Done Done Sent false
23922 7124 Acked Read 5 Done Done Sent false
23923 7125 Acked Write 1 Done Done Sent false
23924 7126 Acked Write 9 Done Done Sent false
23925 7127 Acked Read 3 Done Done Sent false
23926 7128 Acked Read 4 Done Done Sent false
23927 7129 Acked Read 4 Done Done Sent false
23928 7130 Acked Write 3 Done Done Sent false
23929 7131 Acked Write 1 Done Done Sent false
23930 7132 Acked Write 3 Done Done Sent false
23931 7133 Acked Read 7 Done Done Sent false
23932 7134 Acked Read 9 Done Done Sent false
23933 7135 Acked Read 3 Done Done Sent false
23934 7136 Acked Flush 0 Done Done Sent false
23935 7137 Acked Read 7 Done Done Sent false
23936 7138 Acked Write 5 Done Done Sent false
23937 7139 Acked Write 10 Done Done Sent false
23938 7140 Acked Write 9 Done Done Sent false
23939 7141 Acked Write 7 Done Done Sent false
23940 7142 Acked Read 3 Done Done Sent false
23941 7143 Acked Write 7 Done Done Sent false
23942 7144 Acked Read 6 Done Done Sent false
23943 7145 Acked Write 10 Done Done Sent false
23944 7146 Acked Read 6 Done Done Sent false
23945 7147 Acked Read 9 Done Done Sent false
23946 7148 Acked Read 3 Done Done Sent false
23947 7149 Acked Read 10 Done Done Sent false
23948 7150 Acked Read 1 Done Done Sent false
23949 7151 Acked Write 5 Done Done Sent false
23950 7152 Acked Read 7 Done Done Sent false
23951 7153 Acked Read 3 Done Done Sent false
23952 7154 Acked Write 9 Done Done Sent false
23953 7155 Acked Write 10 Done Done Sent false
23954 7156 Acked Read 1 Done Done Sent false
23955 7157 Acked Read 4 Done Done Sent false
23956 7158 Acked Flush 0 Done Done Sent false
23957 7159 Acked Flush 0 Done Done Sent false
23958 7160 Acked Read 5 Done Done Sent false
23959 7161 Acked Write 5 Done Done Sent false
23960 7162 Acked Read 6 Done Done Sent false
23961 7163 Acked Read 4 Done Done Sent false
23962 7164 Acked Flush 0 Done Done Sent false
23963 7165 Acked Read 6 Done Done Sent false
23964 7166 Acked Read 4 Done Done Sent false
23965 7167 Acked Flush 0 Done Done Sent false
23966 7168 Acked Read 5 Done Done Sent false
23967 7169 Acked Write 8 Done Done Sent false
23968 7170 Acked Read 3 Done Done Sent false
23969 7171 Acked Read 10 Done Done Sent false
23970 7172 Acked Read 9 Done Done Sent false
23971 7173 Acked Read 6 Done Done Sent false
23972 7174 Acked Write 6 Done Done Sent false
23973 7175 Acked Write 9 Done Done Sent false
23974 7176 Acked Read 7 Done Done Sent false
23975 7177 Acked Write 1 Done Done Sent false
23976 7178 Acked Read 6 Done Done Sent false
23977 7179 Acked Write 3 Done Done Sent false
23978 7180 Acked Read 2 Done Done Sent false
23979 7181 Acked Write 6 Done Done Sent false
23980 7182 Acked Read 5 Done Done Sent false
23981 7183 Acked Read 2 Done Done Sent false
23982 7184 Acked Read 2 Done Done Sent false
23983 7185 Acked Read 4 Done Done Sent false
23984 7186 Acked Read 5 Done Done Sent false
23985 7187 Acked Read 6 Done Done Sent false
23986 7188 Acked Write 8 Done Done Sent false
23987 7189 Acked Read 4 Done Done Sent false
23988 7190 Acked Read 9 Done Done Sent false
23989 7191 Acked Flush 0 Done Done Sent false
23990 7192 Acked Read 6 Done Done Sent false
23991 7193 Acked Read 9 Done Done Sent false
23992 7194 Acked Flush 0 Done Done Sent false
23993 7195 Acked Read 1 Done Done Sent false
23994 7196 Acked Read 8 Done Done Sent false
23995 7197 Acked Flush 0 Done Done Sent false
23996 7198 Acked Read 9 Done Done Sent false
23997 7199 Acked Read 5 Done Done Sent false
23998 7200 Acked Read 4 Done Done Sent false
23999 7201 Acked Read 5 Done Done Sent false
24000 7202 Acked Write 8 Done Done Sent false
24001 7203 Acked Read 10 Done Done Sent false
24002 7204 Acked Write 7 Done Done Sent false
24003 7205 Acked Read 8 Done Done Sent false
24004 7206 Acked Write 3 Done Done Sent false
24005 7207 Acked Write 1 Done Done Sent false
24006 7208 Acked Read 7 Done Done Sent false
24007 7209 Acked Write 4 Done Done Sent false
24008 7210 Acked Read 10 Done Done Sent false
24009 7211 Acked Write 3 Done Done Sent false
24010 7212 Acked Write 4 Done Done Sent false
24011 7213 Acked Read 3 Done Done Sent false
24012 7214 Acked Write 8 Done Done Sent false
24013 7215 Acked Read 9 Done Done Sent false
24014 7216 Acked Write 1 Done Done Sent false
24015 7217 Acked Flush 0 Done Done Sent false
24016 7218 Acked Write 2 Done Done Sent false
24017 7219 Acked Write 7 Done Done Sent false
24018 7220 Acked Write 5 Done Done Sent false
24019 7221 Acked Write 6 Done Done Sent false
24020 7222 Acked Read 7 Done Done Sent false
24021 7223 Acked Flush 0 Done Done Sent false
24022 7224 Acked Flush 0 Done Done Sent false
24023 7225 Acked Write 3 Done Done Sent false
24024 7226 Acked Write 3 Done Done Sent false
24025 7227 Acked Write 2 Done Done Sent false
24026 7228 Acked Write 10 Done Done Sent false
24027 7229 Acked Read 10 Done Done Sent false
24028 7230 Acked Write 1 Done Done Sent false
24029 7231 Acked Write 2 Done Done Sent false
24030 7232 Acked Read 10 Done Done Sent false
24031 7233 Acked Write 10 Done Done Sent false
24032 7234 Acked Read 9 Done Done Sent false
24033 7235 Acked Write 9 Done Done Sent false
24034 7236 Acked Read 1 Done Done Sent false
24035 7237 Acked Read 2 Done Done Sent false
24036 7238 Acked Write 4 Done Done Sent false
24037 7239 Acked Write 2 Done Done Sent false
24038 7240 Acked Write 7 Done Done Sent false
24039 7241 Acked Write 4 Done Done Sent false
24040 7242 Acked Write 5 Done Done Sent false
24041 7243 Acked Read 2 Done Done Sent false
24042 7244 Acked Read 8 Done Done Sent false
24043 7245 Acked Write 8 Done Done Sent false
24044 7246 Acked Read 8 Done Done Sent false
24045 7247 Acked Write 4 Done Done Sent false
24046 7248 Acked Write 4 Done Done Sent false
24047 7249 Acked Write 4 Done Done Sent false
24048 7250 Acked Write 3 Done Done Sent false
24049 7251 Acked Read 1 Done Done Sent false
24050 7252 Acked Read 8 Done Done Sent false
24051 7253 Acked Read 4 Done Done Sent false
24052 7254 Acked Read 5 Done Done Sent false
24053 7255 Acked Write 1 Done Done Sent false
24054 7256 Acked Read 7 Done Done Sent false
24055 7257 Acked Read 9 Done Done Sent false
24056 7258 Acked Read 8 Done Done Sent false
24057 7259 Acked Read 6 Done Done Sent false
24058 7260 Acked Read 10 Done Done Sent false
24059 7261 Acked Flush 0 Done Done Sent false
24060 7262 Acked Write 8 Done Done Sent false
24061 7263 Acked Read 1 Done Done Sent false
24062 7264 Acked Read 2 Done Done Sent false
24063 7265 Acked Read 10 Done Done Sent false
24064 7266 Acked Write 6 Done Done Sent false
24065 7267 Acked Read 1 Done Done Sent false
24066 7268 Acked Flush 0 Done Done Sent false
24067 7269 Acked Write 3 Done Done Sent false
24068 7270 Acked Read 10 Done Done Sent false
24069 7271 Acked Read 9 Done Done Sent false
24070 7272 Acked Read 10 Done Done Sent false
24071 7273 Acked Read 6 Done Done Sent false
24072 7274 Acked Write 4 Done Done Sent false
24073 7275 Acked Write 8 Done Done Sent false
24074 7276 Acked Read 3 Done Done Sent false
24075 7277 Acked Flush 0 Done Done Sent false
24076 7278 Acked Flush 0 Done Done Sent false
24077 7279 Acked Flush 0 Done Done Sent false
24078 7280 Acked Write 8 Done Done Sent false
24079 7281 Acked Write 4 Done Done Sent false
24080 7282 Acked Flush 0 Done Done Sent false
24081 7283 Acked Read 6 Done Done Sent false
24082 7284 Acked Read 1 Done Done Sent false
24083 7285 Acked Write 2 Done Done Sent false
24084 7286 Acked Flush 0 Done Done Sent false
24085 7287 Acked Write 10 Done Done Sent false
24086 7288 Acked Read 3 Done Done Sent false
24087 7289 Acked Flush 0 Done Done Sent false
24088 7290 Acked Read 4 Done Done Sent false
24089 7291 Acked Write 5 Done Done Sent false
24090 7292 Acked Read 6 Done Done Sent false
24091 7293 Acked Read 1 Done Done Sent false
24092 7294 Acked Write 3 Done Done Sent false
24093 7295 Acked Read 10 Done Done Sent false
24094 7296 Acked Read 4 Done Done Sent false
24095 7297 Acked Read 1 Done Done Sent false
24096 7298 Acked Write 10 Done Done Sent false
24097 7299 Acked Write 3 Done Done Sent false
24098 7300 Acked Write 3 Done Done Sent false
24099 7301 Acked Write 9 Done Done Sent false
24100 7302 Acked Read 2 Done Done Sent false
24101 7303 Acked Write 1 Done Done Sent false
24102 7304 Acked Read 8 Done Done Sent false
24103 7305 Acked Write 9 Done Done Sent false
24104 7306 Acked Read 1 Done Done Sent false
24105 7307 Acked Read 1 Done Done Sent false
24106 7308 Acked Write 8 Done Done Sent false
24107 7309 Acked Write 1 Done Done Sent false
24108 7310 Acked Write 7 Done Done Sent false
24109 7311 Acked Read 5 Done Done Sent false
24110 7312 Acked Write 5 Done Done Sent false
24111 7313 Acked Write 8 Done Done Sent false
24112 7314 Acked Read 3 Done Done Sent false
24113 7315 Acked Read 6 Done Done Sent false
24114 7316 Acked Read 1 Done Done Sent false
24115 7317 Acked Write 3 Done Done Sent false
24116 7318 Acked Write 3 Done Done Sent false
24117 7319 Acked Read 5 Done Done Sent false
24118 7320 Acked Write 7 Done Done Sent false
24119 7321 Acked Write 8 Done Done Sent false
24120 7322 Acked Flush 0 Done Done Sent false
24121 7323 Acked Write 2 Done Done Sent false
24122 7324 Acked Flush 0 Done Done Sent false
24123 7325 Acked Write 1 Done Done Sent false
24124 7326 Acked Read 1 Done Done Sent false
24125 7327 Acked Write 4 Done Done Sent false
24126 7328 Acked Write 6 Done Done Sent false
24127 7329 Acked Read 2 Done Done Sent false
24128 7330 Acked Flush 0 Done Done Sent false
24129 7331 Acked Read 7 Done Done Sent false
24130 7332 Acked Read 7 Done Done Sent false
24131 7333 Acked Write 4 Done Done Sent false
24132 7334 Acked Write 1 Done Done Sent false
24133 7335 Acked Write 10 Done Done Sent false
24134 7336 Acked Write 5 Done Done Sent false
24135 7337 Acked Write 6 Done Done Sent false
24136 7338 Acked Write 4 Done Done Sent false
24137 7339 Acked Write 1 Done Done Sent false
24138 7340 Acked Read 1 Done Done Sent false
24139 7341 Acked Read 4 Done Done Sent false
24140 7342 Acked Read 8 Done Done Sent false
24141 7343 Acked Read 2 Done Done Sent false
24142 7344 Acked Read 2 Done Done Sent false
24143 7345 Acked Read 1 Done Done Sent false
24144 7346 Acked Read 8 Done Done Sent false
24145 7347 Acked Write 8 Done Done Sent false
24146 7348 Acked Write 9 Done Done Sent false
24147 STATES DS:0 DS:1 DS:2 TOTAL
24148 Sent 0 0 302 302
24149 Done 302 302 0 604
24150 Skipped 0 0 0 0
24151 Error 0 0 0 0
24152 Last Flush: 7330 7330 7046
24153 Downstairs last five completed: 7046 7045 7044 7043 7042
24154 Upstairs last five completed: 7348 7347 7346 7345 7344
24155 CLIENT: Up:0 ds:302 act:2
24156 [8830] got start action
24157 [8830] I am going to restart
24158 Starting downstairs at port 8830
24159 Make output file at "/tmp/dsc/downstairs-8830.txt"
24160 [8830][2] reports Starting
24161 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1220)
24162 [8830][2] reports Running
24163 ----------------------------------------------------------------
24164 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
24165 States: Active Active Offline
24166 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24167 7047 Acked Write 3 Done Done Sent false
24168 7048 Acked Read 10 Done Done Sent false
24169 7049 Acked Read 4 Done Done Sent false
24170 7050 Acked Read 2 Done Done Sent false
24171 7051 Acked Read 2 Done Done Sent false
24172 7052 Acked Read 4 Done Done Sent false
24173 7053 Acked Write 7 Done Done Sent false
24174 7054 Acked Write 4 Done Done Sent false
24175 7055 Acked Read 2 Done Done Sent false
24176 7056 Acked Write 2 Done Done Sent false
24177 7057 Acked Read 6 Done Done Sent false
24178 7058 Acked Read 1 Done Done Sent false
24179 7059 Acked Write 10 Done Done Sent false
24180 7060 Acked Write 3 Done Done Sent false
24181 7061 Acked Read 3 Done Done Sent false
24182 7062 Acked Read 3 Done Done Sent false
24183 7063 Acked Read 1 Done Done Sent false
24184 7064 Acked Read 6 Done Done Sent false
24185 7065 Acked Read 8 Done Done Sent false
24186 7066 Acked Read 7 Done Done Sent false
24187 7067 Acked Write 7 Done Done Sent false
24188 7068 Acked Write 9 Done Done Sent false
24189 7069 Acked Write 5 Done Done Sent false
24190 7070 Acked Flush 0 Done Done Sent false
24191 7071 Acked Read 3 Done Done Sent false
24192 7072 Acked Write 4 Done Done Sent false
24193 7073 Acked Read 7 Done Done Sent false
24194 7074 Acked Read 9 Done Done Sent false
24195 7075 Acked Write 5 Done Done Sent false
24196 7076 Acked Write 10 Done Done Sent false
24197 7077 Acked Read 6 Done Done Sent false
24198 7078 Acked Read 4 Done Done Sent false
24199 7079 Acked Read 5 Done Done Sent false
24200 7080 Acked Read 7 Done Done Sent false
24201 7081 Acked Write 10 Done Done Sent false
24202 7082 Acked Read 7 Done Done Sent false
24203 7083 Acked Read 9 Done Done Sent false
24204 7084 Acked Read 1 Done Done Sent false
24205 7085 Acked Write 1 Done Done Sent false
24206 7086 Acked Write 4 Done Done Sent false
24207 7087 Acked Read 6 Done Done Sent false
24208 7088 Acked Write 10 Done Done Sent false
24209 7089 Acked Read 7 Done Done Sent false
24210 7090 Acked Read 5 Done Done Sent false
24211 7091 Acked Write 5 Done Done Sent false
24212 7092 Acked Write 4 Done Done Sent false
24213 7093 Acked Write 2 Done Done Sent false
24214 7094 Acked Write 6 Done Done Sent false
24215 7095 Acked Write 8 Done Done Sent false
24216 7096 Acked Read 8 Done Done Sent false
24217 7097 Acked Write 5 Done Done Sent false
24218 7098 Acked Read 3 Done Done Sent false
24219 7099 Acked Read 6 Done Done Sent false
24220 7100 Acked Read 10 Done Done Sent false
24221 7101 Acked Read 4 Done Done Sent false
24222 7102 Acked Flush 0 Done Done Sent false
24223 7103 Acked Read 2 Done Done Sent false
24224 7104 Acked Read 1 Done Done Sent false
24225 7105 Acked Flush 0 Done Done Sent false
24226 7106 Acked Flush 0 Done Done Sent false
24227 7107 Acked Write 3 Done Done Sent false
24228 7108 Acked Flush 0 Done Done Sent false
24229 7109 Acked Read 2 Done Done Sent false
24230 7110 Acked Read 10 Done Done Sent false
24231 7111 Acked Write 10 Done Done Sent false
24232 7112 Acked Read 5 Done Done Sent false
24233 7113 Acked Write 6 Done Done Sent false
24234 7114 Acked Flush 0 Done Done Sent false
24235 7115 Acked Read 10 Done Done Sent false
24236 7116 Acked Read 4 Done Done Sent false
24237 7117 Acked Read 4 Done Done Sent false
24238 7118 Acked Write 8 Done Done Sent false
24239 7119 Acked Write 10 Done Done Sent false
24240 7120 Acked Write 4 Done Done Sent false
24241 7121 Acked Read 9 Done Done Sent false
24242 7122 Acked Read 5 Done Done Sent false
24243 7123 Acked Write 3 Done Done Sent false
24244 7124 Acked Read 5 Done Done Sent false
24245 7125 Acked Write 1 Done Done Sent false
24246 7126 Acked Write 9 Done Done Sent false
24247 7127 Acked Read 3 Done Done Sent false
24248 7128 Acked Read 4 Done Done Sent false
24249 7129 Acked Read 4 Done Done Sent false
24250 7130 Acked Write 3 Done Done Sent false
24251 7131 Acked Write 1 Done Done Sent false
24252 7132 Acked Write 3 Done Done Sent false
24253 7133 Acked Read 7 Done Done Sent false
24254 7134 Acked Read 9 Done Done Sent false
24255 7135 Acked Read 3 Done Done Sent false
24256 7136 Acked Flush 0 Done Done Sent false
24257 7137 Acked Read 7 Done Done Sent false
24258 7138 Acked Write 5 Done Done Sent false
24259 7139 Acked Write 10 Done Done Sent false
24260 7140 Acked Write 9 Done Done Sent false
24261 7141 Acked Write 7 Done Done Sent false
24262 7142 Acked Read 3 Done Done Sent false
24263 7143 Acked Write 7 Done Done Sent false
24264 7144 Acked Read 6 Done Done Sent false
24265 7145 Acked Write 10 Done Done Sent false
24266 7146 Acked Read 6 Done Done Sent false
24267 7147 Acked Read 9 Done Done Sent false
24268 7148 Acked Read 3 Done Done Sent false
24269 7149 Acked Read 10 Done Done Sent false
24270 7150 Acked Read 1 Done Done Sent false
24271 7151 Acked Write 5 Done Done Sent false
24272 7152 Acked Read 7 Done Done Sent false
24273 7153 Acked Read 3 Done Done Sent false
24274 7154 Acked Write 9 Done Done Sent false
24275 7155 Acked Write 10 Done Done Sent false
24276 7156 Acked Read 1 Done Done Sent false
24277 7157 Acked Read 4 Done Done Sent false
24278 7158 Acked Flush 0 Done Done Sent false
24279 7159 Acked Flush 0 Done Done Sent false
24280 7160 Acked Read 5 Done Done Sent false
24281 7161 Acked Write 5 Done Done Sent false
24282 7162 Acked Read 6 Done Done Sent false
24283 7163 Acked Read 4 Done Done Sent false
24284 7164 Acked Flush 0 Done Done Sent false
24285 7165 Acked Read 6 Done Done Sent false
24286 7166 Acked Read 4 Done Done Sent false
24287 7167 Acked Flush 0 Done Done Sent false
24288 7168 Acked Read 5 Done Done Sent false
24289 7169 Acked Write 8 Done Done Sent false
24290 7170 Acked Read 3 Done Done Sent false
24291 7171 Acked Read 10 Done Done Sent false
24292 7172 Acked Read 9 Done Done Sent false
24293 7173 Acked Read 6 Done Done Sent false
24294 7174 Acked Write 6 Done Done Sent false
24295 7175 Acked Write 9 Done Done Sent false
24296 7176 Acked Read 7 Done Done Sent false
24297 7177 Acked Write 1 Done Done Sent false
24298 7178 Acked Read 6 Done Done Sent false
24299 7179 Acked Write 3 Done Done Sent false
24300 7180 Acked Read 2 Done Done Sent false
24301 7181 Acked Write 6 Done Done Sent false
24302 7182 Acked Read 5 Done Done Sent false
24303 7183 Acked Read 2 Done Done Sent false
24304 7184 Acked Read 2 Done Done Sent false
24305 7185 Acked Read 4 Done Done Sent false
24306 7186 Acked Read 5 Done Done Sent false
24307 7187 Acked Read 6 Done Done Sent false
24308 7188 Acked Write 8 Done Done Sent false
24309 7189 Acked Read 4 Done Done Sent false
24310 7190 Acked Read 9 Done Done Sent false
24311 7191 Acked Flush 0 Done Done Sent false
24312 7192 Acked Read 6 Done Done Sent false
24313 7193 Acked Read 9 Done Done Sent false
24314 7194 Acked Flush 0 Done Done Sent false
24315 7195 Acked Read 1 Done Done Sent false
24316 7196 Acked Read 8 Done Done Sent false
24317 7197 Acked Flush 0 Done Done Sent false
24318 7198 Acked Read 9 Done Done Sent false
24319 7199 Acked Read 5 Done Done Sent false
24320 7200 Acked Read 4 Done Done Sent false
24321 7201 Acked Read 5 Done Done Sent false
24322 7202 Acked Write 8 Done Done Sent false
24323 7203 Acked Read 10 Done Done Sent false
24324 7204 Acked Write 7 Done Done Sent false
24325 7205 Acked Read 8 Done Done Sent false
24326 7206 Acked Write 3 Done Done Sent false
24327 7207 Acked Write 1 Done Done Sent false
24328 7208 Acked Read 7 Done Done Sent false
24329 7209 Acked Write 4 Done Done Sent false
24330 7210 Acked Read 10 Done Done Sent false
24331 7211 Acked Write 3 Done Done Sent false
24332 7212 Acked Write 4 Done Done Sent false
24333 7213 Acked Read 3 Done Done Sent false
24334 7214 Acked Write 8 Done Done Sent false
24335 7215 Acked Read 9 Done Done Sent false
24336 7216 Acked Write 1 Done Done Sent false
24337 7217 Acked Flush 0 Done Done Sent false
24338 7218 Acked Write 2 Done Done Sent false
24339 7219 Acked Write 7 Done Done Sent false
24340 7220 Acked Write 5 Done Done Sent false
24341 7221 Acked Write 6 Done Done Sent false
24342 7222 Acked Read 7 Done Done Sent false
24343 7223 Acked Flush 0 Done Done Sent false
24344 7224 Acked Flush 0 Done Done Sent false
24345 7225 Acked Write 3 Done Done Sent false
24346 7226 Acked Write 3 Done Done Sent false
24347 7227 Acked Write 2 Done Done Sent false
24348 7228 Acked Write 10 Done Done Sent false
24349 7229 Acked Read 10 Done Done Sent false
24350 7230 Acked Write 1 Done Done Sent false
24351 7231 Acked Write 2 Done Done Sent false
24352 7232 Acked Read 10 Done Done Sent false
24353 7233 Acked Write 10 Done Done Sent false
24354 7234 Acked Read 9 Done Done Sent false
24355 7235 Acked Write 9 Done Done Sent false
24356 7236 Acked Read 1 Done Done Sent false
24357 7237 Acked Read 2 Done Done Sent false
24358 7238 Acked Write 4 Done Done Sent false
24359 7239 Acked Write 2 Done Done Sent false
24360 7240 Acked Write 7 Done Done Sent false
24361 7241 Acked Write 4 Done Done Sent false
24362 7242 Acked Write 5 Done Done Sent false
24363 7243 Acked Read 2 Done Done Sent false
24364 7244 Acked Read 8 Done Done Sent false
24365 7245 Acked Write 8 Done Done Sent false
24366 7246 Acked Read 8 Done Done Sent false
24367 7247 Acked Write 4 Done Done Sent false
24368 7248 Acked Write 4 Done Done Sent false
24369 7249 Acked Write 4 Done Done Sent false
24370 7250 Acked Write 3 Done Done Sent false
24371 7251 Acked Read 1 Done Done Sent false
24372 7252 Acked Read 8 Done Done Sent false
24373 7253 Acked Read 4 Done Done Sent false
24374 7254 Acked Read 5 Done Done Sent false
24375 7255 Acked Write 1 Done Done Sent false
24376 7256 Acked Read 7 Done Done Sent false
24377 7257 Acked Read 9 Done Done Sent false
24378 7258 Acked Read 8 Done Done Sent false
24379 7259 Acked Read 6 Done Done Sent false
24380 7260 Acked Read 10 Done Done Sent false
24381 7261 Acked Flush 0 Done Done Sent false
24382 7262 Acked Write 8 Done Done Sent false
24383 7263 Acked Read 1 Done Done Sent false
24384 7264 Acked Read 2 Done Done Sent false
24385 7265 Acked Read 10 Done Done Sent false
24386 7266 Acked Write 6 Done Done Sent false
24387 7267 Acked Read 1 Done Done Sent false
24388 7268 Acked Flush 0 Done Done Sent false
24389 7269 Acked Write 3 Done Done Sent false
24390 7270 Acked Read 10 Done Done Sent false
24391 7271 Acked Read 9 Done Done Sent false
24392 7272 Acked Read 10 Done Done Sent false
24393 7273 Acked Read 6 Done Done Sent false
24394 7274 Acked Write 4 Done Done Sent false
24395 7275 Acked Write 8 Done Done Sent false
24396 7276 Acked Read 3 Done Done Sent false
24397 7277 Acked Flush 0 Done Done Sent false
24398 7278 Acked Flush 0 Done Done Sent false
24399 7279 Acked Flush 0 Done Done Sent false
24400 7280 Acked Write 8 Done Done Sent false
24401 7281 Acked Write 4 Done Done Sent false
24402 7282 Acked Flush 0 Done Done Sent false
24403 7283 Acked Read 6 Done Done Sent false
24404 7284 Acked Read 1 Done Done Sent false
24405 7285 Acked Write 2 Done Done Sent false
24406 7286 Acked Flush 0 Done Done Sent false
24407 7287 Acked Write 10 Done Done Sent false
24408 7288 Acked Read 3 Done Done Sent false
24409 7289 Acked Flush 0 Done Done Sent false
24410 7290 Acked Read 4 Done Done Sent false
24411 7291 Acked Write 5 Done Done Sent false
24412 7292 Acked Read 6 Done Done Sent false
24413 7293 Acked Read 1 Done Done Sent false
24414 7294 Acked Write 3 Done Done Sent false
24415 7295 Acked Read 10 Done Done Sent false
24416 7296 Acked Read 4 Done Done Sent false
24417 7297 Acked Read 1 Done Done Sent false
24418 7298 Acked Write 10 Done Done Sent false
24419 7299 Acked Write 3 Done Done Sent false
24420 7300 Acked Write 3 Done Done Sent false
24421 7301 Acked Write 9 Done Done Sent false
24422 7302 Acked Read 2 Done Done Sent false
24423 7303 Acked Write 1 Done Done Sent false
24424 7304 Acked Read 8 Done Done Sent false
24425 7305 Acked Write 9 Done Done Sent false
24426 7306 Acked Read 1 Done Done Sent false
24427 7307 Acked Read 1 Done Done Sent false
24428 7308 Acked Write 8 Done Done Sent false
24429 7309 Acked Write 1 Done Done Sent false
24430 7310 Acked Write 7 Done Done Sent false
24431 7311 Acked Read 5 Done Done Sent false
24432 7312 Acked Write 5 Done Done Sent false
24433 7313 Acked Write 8 Done Done Sent false
24434 7314 Acked Read 3 Done Done Sent false
24435 7315 Acked Read 6 Done Done Sent false
24436 7316 Acked Read 1 Done Done Sent false
24437 7317 Acked Write 3 Done Done Sent false
24438 7318 Acked Write 3 Done Done Sent false
24439 7319 Acked Read 5 Done Done Sent false
24440 7320 Acked Write 7 Done Done Sent false
24441 7321 Acked Write 8 Done Done Sent false
24442 7322 Acked Flush 0 Done Done Sent false
24443 7323 Acked Write 2 Done Done Sent false
24444 7324 Acked Flush 0 Done Done Sent false
24445 7325 Acked Write 1 Done Done Sent false
24446 7326 Acked Read 1 Done Done Sent false
24447 7327 Acked Write 4 Done Done Sent false
24448 7328 Acked Write 6 Done Done Sent false
24449 7329 Acked Read 2 Done Done Sent false
24450 7330 Acked Flush 0 Done Done Sent false
24451 7331 Acked Read 7 Done Done Sent false
24452 7332 Acked Read 7 Done Done Sent false
24453 7333 Acked Write 4 Done Done Sent false
24454 7334 Acked Write 1 Done Done Sent false
24455 7335 Acked Write 10 Done Done Sent false
24456 7336 Acked Write 5 Done Done Sent false
24457 7337 Acked Write 6 Done Done Sent false
24458 7338 Acked Write 4 Done Done Sent false
24459 7339 Acked Write 1 Done Done Sent false
24460 7340 Acked Read 1 Done Done Sent false
24461 7341 Acked Read 4 Done Done Sent false
24462 7342 Acked Read 8 Done Done Sent false
24463 7343 Acked Read 2 Done Done Sent false
24464 7344 Acked Read 2 Done Done Sent false
24465 7345 Acked Read 1 Done Done Sent false
24466 7346 Acked Read 8 Done Done Sent false
24467 7347 Acked Write 8 Done Done Sent false
24468 7348 Acked Write 9 Done Done Sent false
24469 7349 Acked Flush 0 Done Done Sent false
24470 STATES DS:0 DS:1 DS:2 TOTAL
24471 Sent 0 0 303 303
24472 Done 303 303 0 606
24473 Skipped 0 0 0 0
24474 Error 0 0 0 0
24475 Last Flush: 7349 7349 7046
24476 Downstairs last five completed: 7046 7045 7044 7043 7042
24477 Upstairs last five completed: 7349 7348 7347 7346 7345
24478 CLIENT: Up:0 ds:303 act:2
244792024-11-07T14:08:25.734ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
244802024-11-07T14:08:25.734ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34891) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
244812024-11-07T14:08:25.735ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
244822024-11-07T14:08:25.735ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
244832024-11-07T14:08:25.735ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
244842024-11-07T14:08:25.735ZINFOcrucible: send last flush ID to this DS: 7046 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
244852024-11-07T14:08:25.736ZINFOcrucible: Replied this last flush ID: 7046 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
244862024-11-07T14:08:25.736ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
244872024-11-07T14:08:25.736ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
244882024-11-07T14:08:25.736ZINFOcrucible: [2] client re-new 303 jobs since flush 7046 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
244892024-11-07T14:08:25.736ZINFOcrucible: [2] Replayed 303 jobs since flush: 7046 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
24490 ----------------------------------------------------------------
24491 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
24492 Downstairs last five completed: 7349 7348 7347 7346 7345
24493 Upstairs last five completed: 7349 7348 7347 7346 7345
24494 CLIENT: Up:0 ds:0 act:3
24495 Replay: All jobs finished
24496 Main task has work to do, go find it
24497 got dsc Stop(2)
24498 stop 2
24499 Nov 07 14:08:28.013 INFO [8830] Got stop action so:false kr:false
24500 request completed, latency_us: 350, response_code: 204, uri: /stop/cid/2, method: POST, req_id: c3fcf569-e7a3-43b7-82cd-e01ba90cfc87, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
245012024-11-07T14:08:28.014ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
245022024-11-07T14:08:28.014ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
245032024-11-07T14:08:28.014ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
245042024-11-07T14:08:28.014ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
24505 Nov 07 14:08:28.014 INFO request completed, latency_us{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:08:28.014295455Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"client":"2","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
24506 : 179, response_code: 200, uri: /state/cid/2, method: GET, req_id: 1531e1e9-5235-4726-bbe5-1622df99441f, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
245072024-11-07T14:08:28.014ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
24508 [8830] Exited with: ExitStatus(unix_wait_status(9))
24509 [8830][2] reports Exit
24510 Nov 07 14:08:32.016 INFO request completed, latency_us: 209, response_code: 200, uri: /state/cid/2, method: GET, req_id: 5747f79d-0b9c-4a68-a71a-1ccc7ff02ed1, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
24511 001/300 Flush
24512 002/300 Write block 1639 len 24576 data: 6 6 5 5 4 5
24513 003/300 Read block 3599 len 20480
24514 004/300 Read block 952 len 4096
24515 005/300 Read block 950 len 24576
24516 006/300 Read block 3252 len 16384
24517 007/300 Write block 1289 len 8192 data: 5 5
24518 008/300 Write block 2212 len 28672 data: 4 3 3 2 2 2 2
24519 009/300 Write block 1271 len 20480 data: 5 5 5 4 4
24520 010/300 Read block 3849 len 20480
24521 011/300 Write block 3332 len 8192 data: 8 7
24522 012/300 Read block 4982 len 36864
24523 013/300 Write block 1852 len 16384 data: 8 8 8 7
24524 014/300 Read block 212 len 24576
24525 015/300 Write block 4248 len 16384 data: 3 4 5 6
24526 016/300 Read block 2190 len 40960
24527 017/300 Read block 4687 len 4096
24528 018/300 Read block 3315 len 32768
24529 019/300 Flush
24530 020/300 Write block 484 len 40960 data: 6 7 6 7 7 3 2 2 2 2
24531 021/300 Read block 2889 len 32768
24532 022/300 Write block 1530 len 28672 data: 3 3 3 3 3 2 2
24533 023/300 Write block 1493 len 20480 data: 6 6 4 4 4
24534 024/300 Read block 415 len 8192
24535 025/300 Read block 1129 len 28672
24536 026/300 Write block 947 len 24576 data: 4 2 4 4 6 6
24537 027/300 Read block 3588 len 8192
24538 028/300 Read block 1800 len 32768
24539 029/300 Write block 3731 len 12288 data: 6 6 5
24540 030/300 Read block 2465 len 28672
24541 031/300 Read block 1273 len 20480
24542 032/300 Write block 57 len 32768 data: 4 3 3 3 3 4 3 3
24543 033/300 Write block 261 len 20480 data: 6 6 7 5 6
24544 034/300 Read block 3064 len 36864
24545 035/300 Read block 1147 len 16384
24546 036/300 Flush
24547 037/300 Write block 1554 len 8192 data: 4 4
24548 038/300 Write block 324 len 24576 data: 5 6 5 5 5 5
24549 039/300 Write block 2013 len 40960 data: 9 8 7 6 6 6 5 5 4 4
24550 040/300 Read block 3519 len 40960
24551 041/300 Write block 4727 len 16384 data: 6 7 8 8
24552 042/300 Read block 3457 len 12288
24553 043/300 Read block 116 len 4096
24554 044/300 Flush
24555 045/300 Write block 2302 len 8192 data: 5 6
24556 046/300 Write block 3072 len 20480 data: 3 3 3 3 5
24557 047/300 Read block 2274 len 16384
24558 048/300 Read block 2927 len 32768
24559 049/300 Read block 4802 len 12288
24560 050/300 Read block 868 len 40960
24561 051/300 Read block 3787 len 32768
24562 052/300 Read block 278 len 40960
24563 053/300 Read block 1779 len 12288
24564 054/300 Read block 2631 len 24576
24565 055/300 Write block 862 len 32768 data: 5 5 6 5 4 4 4 4
24566 056/300 Write block 4931 len 32768 data: 5 4 5 4 5 4 3 4
24567 057/300 Write block 3976 len 16384 data: 4 5 4 4
24568 058/300 Read block 3165 len 40960
24569 059/300 Write block 1141 len 4096 data: 5
24570 060/300 Write block 3099 len 28672 data: 2 2 2 3 3 4 7
24571 061/300 Read block 2857 len 16384
24572 062/300 Read block 84 len 16384
24573 063/300 Write block 1883 len 20480 data: 6 7 6 5 5
24574 064/300 Read block 3911 len 32768
24575 065/300 Read block 3722 len 28672
24576 066/300 Write block 4437 len 32768 data: 7 8 9 7 8 7 6 6
24577 067/300 Read block 3327 len 36864
24578 068/300 Write block 621 len 40960 data: 5 6 6 4 4 5 4 3 3 3
24579 069/300 Read block 1153 len 40960
24580 070/300 Read block 3296 len 12288
24581 071/300 Read block 1294 len 24576
24582 072/300 Read block 1179 len 32768
24583 073/300 Flush
24584 074/300 Read block 4016 len 32768
24585 075/300 Read block 1034 len 12288
24586 076/300 Write block 1754 len 36864 data: 6 5 5 5 4 4 3 3 5
24587 077/300 Write block 2939 len 20480 data: 5 4 4 6 6
24588 078/300 Read block 375 len 40960
24589 079/300 Read block 3368 len 8192
24590 080/300 Write block 1299 len 8192 data: 3 3
24591 081/300 Write block 3971 len 20480 data: 7 9 7 5 4
24592 082/300 Write block 4591 len 28672 data: 5 5 5 6 6 6 5
24593 083/300 Write block 1995 len 32768 data: 3 3 4 5 5 5 5 5
24594 084/300 Write block 4795 len 24576 data: 5 5 5 5 6 5
24595 085/300 Read block 4522 len 28672
24596 086/300 Write block 1255 len 4096 data: 5
24597 087/300 Write block 1484 len 32768 data: 6 6 5 5 5 5 4 4
24598 088/300 Write block 3870 len 4096 data: 7
24599 089/300 Read block 95 len 36864
24600 090/300 Read block 1757 len 12288
24601 091/300 Read block 4820 len 4096
24602 092/300 Write block 256 len 4096 data: 6
24603 093/300 Write block 4875 len 16384 data: 4 4 4 3
24604 094/300 Read block 202 len 20480
24605 095/300 Write block 3806 len 36864 data: 3 3 4 3 3 3 3 5 6
24606 096/300 Read block 483 len 12288
24607 097/300 Read block 1387 len 36864
24608 098/300 Flush
24609 099/300 Write block 4455 len 4096 data: 3
24610 100/300 Read block 693 len 32768
24611 101/300 Read block 4433 len 16384
24612 102/300 Read block 2626 len 12288
24613 103/300 Write block 2580 len 32768 data: 4 2 3 3 3 3 3 3
24614 104/300 Read block 4964 len 4096
24615 105/300 Write block 711 len 36864 data: 7 7 7 5 4 3 2 2 3
24616 106/300 Write block 587 len 40960 data: 3 2 3 3 3 3 6 5 4 3
24617 107/300 Read block 1341 len 36864
24618 108/300 Read block 443 len 28672
24619 109/300 Write block 4862 len 24576 data: 3 3 5 3 3 4
24620 110/300 Read block 4251 len 12288
24621 111/300 Read block 4647 len 32768
24622 112/300 Read block 4833 len 20480
24623 113/300 Write block 4662 len 8192 data: 5 4
24624 114/300 Read block 767 len 4096
24625 115/300 Flush
24626 116/300 Flush
24627 117/300 Flush
24628 118/300 Read block 4880 len 20480
24629 119/300 Write block 1320 len 36864 data: 6 7 7 4 6 8 7 7 8
24630 120/300 Read block 4007 len 4096
24631 121/300 Write block 1557 len 20480 data: 4 4 3 5 5
24632 122/300 Write block 2021 len 28672 data: 5 5 3 3 3 4 3
24633 123/300 Read block 3158 len 28672
24634 124/300 Read block 3574 len 20480
24635 125/300 Write block 2912 len 12288 data: 8 7 7
24636 126/300 Read block 1327 len 20480
24637 127/300 Read block 2641 len 40960
24638 128/300 Read block 4884 len 20480
24639 129/300 Read block 113 len 24576
24640 130/300 Write block 1350 len 28672 data: 5 4 3 6 8 7 10
24641 131/300 Flush
24642 132/300 Write block 1903 len 32768 data: 5 5 4 4 3 4 4 4
24643 133/300 Read block 3567 len 24576
24644 134/300 Write block 3721 len 12288 data: 9 10 8
24645 135/300 Write block 2355 len 16384 data: 3 3 4 4
24646 136/300 Write block 2756 len 16384 data: 4 4 3 2
24647 137/300 Read block 1657 len 16384
24648 138/300 Write block 123 len 12288 data: 4 4 3
24649 139/300 Read block 935 len 12288
24650 140/300 Read block 1895 len 16384
24651 141/300 Read block 3076 len 36864
24652 142/300 Read block 3505 len 16384
24653 143/300 Write block 4157 len 8192 data: 3 2
24654 144/300 Write block 323 len 40960 data: 3 6 7 6 6 6 6 5 5 5
24655 145/300 Read block 4161 len 24576
24656 146/300 Write block 3525 len 40960 data: 2 2 2 2 2 3 4 4 6 6
24657 147/300 Read block 1261 len 8192
24658 148/300 Flush
24659 149/300 Read block 2332 len 16384
24660 150/300 Read block 3273 len 32768
24661 151/300 Read block 4379 len 4096
24662 152/300 Read block 703 len 36864
24663 153/300 Read block 1777 len 40960
24664 154/300 Read block 109 len 20480
24665 155/300 Read block 3237 len 32768
24666 156/300 Write block 3443 len 36864 data: 6 5 5 5 4 4 3 4 4
24667 157/300 Write block 4951 len 40960 data: 5 6 8 9 10 8 9 8 8 8
24668 158/300 Write block 3363 len 12288 data: 3 3 3
24669 159/300 Write block 17 len 4096 data: 4
24670 160/300 Write block 3821 len 36864 data: 4 5 5 4 3 4 4 7 7
24671 161/300 Write block 3130 len 40960 data: 6 6 5 5 5 6 6 3 3 2
24672 162/300 Flush
24673 163/300 Read block 4298 len 8192
24674 164/300 Read block 4781 len 36864
24675 165/300 Write block 330 len 20480 data: 6 6 6 6 5
24676 166/300 Write block 3361 len 16384 data: 3 3 4 4
24677 167/300 Read block 283 len 16384
24678 168/300 Read block 2942 len 12288
24679 169/300 Write block 1630 len 8192 data: 3 5
24680 170/300 Write block 2074 len 16384 data: 2 3 4 4
24681 171/300 Read block 4146 len 20480
24682 172/300 Read block 2571 len 28672
24683 173/300 Read block 2831 len 28672
24684 174/300 Write block 1205 len 32768 data: 4 5 4 3 3 4 5 4
24685 175/300 Write block 3493 len 28672 data: 6 6 6 2 2 2 2
24686 176/300 Write block 3175 len 28672 data: 4 5 5 6 4 3 4
24687 177/300 Read block 3551 len 8192
24688 178/300 Read block 4812 len 24576
24689 179/300 Write block 4084 len 36864 data: 2 3 3 3 3 3 3 3 3
24690 180/300 Read block 753 len 8192
24691 181/300 Read block 410 len 8192
24692 182/300 Write block 3705 len 8192 data: 5 4
24693 183/300 Write block 1164 len 28672 data: 4 4 2 3 4 4 4
24694 184/300 Read block 3622 len 24576
24695 185/300 Read block 4010 len 12288
24696 186/300 Write block 1780 len 32768 data: 5 3 4 4 4 3 4 4
24697 187/300 Write block 1685 len 24576 data: 3 3 2 2 2 3
24698 188/300 Flush
24699 189/300 Write block 2248 len 20480 data: 4 5 4 5 5
24700 190/300 Write block 2905 len 36864 data: 4 5 5 5 6 6 7 9 8
24701 191/300 Read block 3291 len 36864
24702 192/300 Flush
24703 193/300 Write block 3836 len 40960 data: 6 5 5 4 4 4 5 5 5 3
24704 194/300 Write block 407 len 28672 data: 7 7 6 8 8 5 5
24705 195/300 Write block 604 len 40960 data: 3 4 4 3 2 3 4 4 5 5
24706 196/300 Read block 2673 len 36864
24707 197/300 Read block 1818 len 8192
24708 198/300 Flush
24709 199/300 Read block 412 len 4096
24710 200/300 Read block 2803 len 36864
24711 201/300 Write block 1483 len 16384 data: 6 7 7 6
24712 202/300 Read block 785 len 4096
24713 203/300 Read block 2066 len 16384
24714 204/300 Read block 373 len 20480
24715 205/300 Flush
24716 206/300 Read block 4272 len 12288
24717 207/300 Write block 4760 len 12288 data: 5 4 4
24718 208/300 Read block 4309 len 16384
24719 209/300 Write block 3710 len 8192 data: 6 8
24720 210/300 Read block 4538 len 16384
24721 211/300 Write block 1047 len 16384 data: 7 6 6 7
24722 212/300 Read block 2678 len 4096
24723 213/300 Read block 3511 len 4096
24724 214/300 Write block 3813 len 4096 data: 6
24725 215/300 Read block 3635 len 4096
24726 216/300 Read block 3870 len 40960
24727 217/300 Write block 3925 len 12288 data: 6 6 7
24728 218/300 Write block 556 len 24576 data: 3 5 4 4 4 4
24729 219/300 Read block 4903 len 20480
24730 220/300 Read block 3623 len 24576
24731 221/300 Read block 1254 len 16384
24732 222/300 Flush
24733 223/300 Read block 4992 len 32768
24734 224/300 Write block 3890 len 24576 data: 7 6 4 3 3 5
24735 225/300 Read block 4857 len 24576
24736 226/300 Read block 1767 len 36864
24737 227/300 Write block 1798 len 32768 data: 3 3 3 3 3 4 4 5
24738 228/300 Write block 2629 len 12288 data: 3 3 5
24739 229/300 Write block 1594 len 40960 data: 4 3 4 6 7 8 8 9 8 8
24740 230/300 Write block 4598 len 4096 data: 4
24741 231/300 Read block 425 len 20480
24742 232/300 Flush
24743 233/300 Write block 3035 len 4096 data: 4
24744 234/300 Write block 125 len 24576 data: 4 4 3 3 3 4
24745 235/300 Read block 436 len 32768
24746 236/300 Read block 1026 len 28672
24747 237/300 Write block 2936 len 40960 data: 4 5 4 6 5 5 7 7 6 6
24748 238/300 Read block 1334 len 40960
24749 239/300 Flush
24750 240/300 Write block 970 len 32768 data: 3 4 3 3 3 4 5 5
24751 241/300 Read block 97 len 16384
24752 242/300 Read block 4830 len 20480
24753 243/300 Read block 3004 len 16384
24754 244/300 Write block 3797 len 40960 data: 7 7 7 7 7 6 7 7 5 4
24755 245/300 Write block 1494 len 32768 data: 7 5 5 5 5 5 3 3
24756 246/300 Write block 2168 len 40960 data: 7 7 6 5 4 2 2 2 2 3
24757 247/300 Read block 3420 len 40960
24758 248/300 Flush
24759 249/300 Read block 830 len 36864
24760 250/300 Write block 854 len 32768 data: 5 5 5 5 4 5 6 6
24761 251/300 Read block 2522 len 40960
24762 252/300 Read block 4254 len 16384
24763 253/300 Read block 2055 len 28672
24764 254/300 Write block 2670 len 16384 data: 6 6 5 3
24765 255/300 Write block 971 len 12288 data: 5 4 4
24766 256/300 Write block 1826 len 12288 data: 4 5 7
24767 257/300 Write block 2113 len 40960 data: 4 4 4 3 3 3 3 3 3 2
24768 258/300 Write block 1667 len 20480 data: 8 6 7 7 5
24769 259/300 Flush
24770 260/300 Read block 2793 len 4096
24771 261/300 Read block 424 len 28672
24772 262/300 Read block 4608 len 36864
24773 263/300 Write block 4792 len 28672 data: 5 4 5 6 6 6 6
24774 264/300 Read block 3221 len 32768
24775 265/300 Read block 1753 len 40960
24776 266/300 Read block 696 len 32768
24777 267/300 Read block 4904 len 12288
24778 268/300 Read block 3257 len 36864
24779 269/300 Read block 412 len 24576
24780 270/300 Read block 3579 len 12288
24781 271/300 Write block 4836 len 24576 data: 3 4 4 3 3 6
24782 272/300 Write block 3187 len 24576 data: 3 3 3 2 2 2
24783 273/300 Write block 4626 len 4096 data: 3
24784 274/300 Read block 1068 len 32768
24785 275/300 Read block 4295 len 16384
24786 276/300 Flush
24787 277/300 Write block 4117 len 32768 data: 4 5 7 6 7 8 7 8
24788 278/300 Flush
24789 279/300 Write block 456 len 8192 data: 7 7
24790 280/300 Read block 1665 len 24576
24791 281/300 Read block 3105 len 8192
24792 282/300 Read block 2388 len 16384
24793 283/300 Read block 1072 len 32768
24794 284/300 Read block 57 len 36864
24795 285/300 Read block 3955 len 8192
24796 286/300 Write block 594 len 28672 data: 6 5 4 3 4 3 3
24797 287/300 Read block 2145 len 32768
24798 288/300 Read block 3929 len 8192
24799 289/300 Flush
24800 290/300 Write block 3080 len 40960 data: 5 8 7 9 9 8 5 4 5 5
24801 291/300 Write block 1046 len 32768 data: 7 8 7 7 8 7 6 5
24802 292/300 Read block 2630 len 16384
24803 293/300 Write block 2320 len 12288 data: 3 4 4
24804 294/300 Read block 2359 len 20480
24805 295/300 Flush
24806 296/300 Read block 2278 len 12288
24807 297/300 Flush
24808 298/300 Write block 4065 len 20480 data: 5 4 4 4 4
24809 299/300 Write block 2024 len 36864 data: 4 4 5 4 5 5 4 4 5
24810 300/300 Write block 4185 len 32768 data: 4 3 2 4 4 4 3 4
24811 301/300 Write block 2669 len 28672 data: 5 7 7 6 4 3 4
24812 Main task has work to do, go find it
24813 got dsc Start(2)
24814 start 2
24815 Nov 07 14:08:32.312 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/2, method: POST, req_id: d9cf434a-16fb-412b-b5ec-9ce57e758499, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
24816 Replay: started 2, returned:Ok(())
24817 ----------------------------------------------------------------
24818 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24819 States: Active Active Offline
24820 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24821 7350 Acked Flush 0 Done Done Sent false
24822 7351 Acked Write 6 Done Done Sent false
24823 7352 Acked Read 5 Done Done Sent false
24824 7353 Acked Read 1 Done Done Sent false
24825 7354 Acked Read 6 Done Done Sent false
24826 7355 Acked Read 4 Done Done Sent false
24827 7356 Acked Write 2 Done Done Sent false
24828 7357 Acked Write 7 Done Done Sent false
24829 7358 Acked Write 5 Done Done Sent false
24830 7359 Acked Read 5 Done Done Sent false
24831 7360 Acked Write 2 Done Done Sent false
24832 7361 Acked Read 9 Done Done Sent false
24833 7362 Acked Write 4 Done Done Sent false
24834 7363 Acked Read 6 Done Done Sent false
24835 7364 Acked Write 4 Done Done Sent false
24836 7365 Acked Read 10 Done Done Sent false
24837 7366 Acked Read 1 Done Done Sent false
24838 7367 Acked Read 8 Done Done Sent false
24839 7368 Acked Flush 0 Done Done Sent false
24840 7369 Acked Write 10 Done Done Sent false
24841 7370 Acked Read 8 Done Done Sent false
24842 7371 Acked Write 7 Done Done Sent false
24843 7372 Acked Write 5 Done Done Sent false
24844 7373 Acked Read 2 Done Done Sent false
24845 7374 Acked Read 7 Done Done Sent false
24846 7375 Acked Write 6 Done Done Sent false
24847 7376 Acked Read 2 Done Done Sent false
24848 7377 Acked Read 8 Done Done Sent false
24849 7378 Acked Write 3 Done Done Sent false
24850 7379 Acked Read 7 Done Done Sent false
24851 7380 Acked Read 5 Done Done Sent false
24852 7381 Acked Write 8 Done Done Sent false
24853 7382 Acked Write 5 Done Done Sent false
24854 7383 Acked Read 9 Done Done Sent false
24855 7384 Acked Read 4 Done Done Sent false
24856 7385 Acked Flush 0 Done Done Sent false
24857 7386 Acked Write 2 Done Done Sent false
24858 7387 Acked Write 6 Done Done Sent false
24859 7388 Acked Write 10 Done Done Sent false
24860 7389 Acked Read 10 Done Done Sent false
24861 7390 Acked Write 4 Done Done Sent false
24862 7391 Acked Read 3 Done Done Sent false
24863 7392 Acked Read 1 Done Done Sent false
24864 7393 Acked Flush 0 Done Done Sent false
24865 7394 Acked Write 2 Done Done Sent false
24866 7395 Acked Write 5 Done Done Sent false
24867 7396 Acked Read 4 Done Done Sent false
24868 7397 Acked Read 8 Done Done Sent false
24869 7398 Acked Read 3 Done Done Sent false
24870 7399 Acked Read 10 Done Done Sent false
24871 7400 Acked Read 8 Done Done Sent false
24872 7401 Acked Read 10 Done Done Sent false
24873 7402 Acked Read 3 Done Done Sent false
24874 7403 Acked Read 6 Done Done Sent false
24875 7404 Acked Write 8 Done Done Sent false
24876 7405 Acked Write 8 Done Done Sent false
24877 7406 Acked Write 4 Done Done Sent false
24878 7407 Acked Read 10 Done Done Sent false
24879 7408 Acked Write 1 Done Done Sent false
24880 7409 Acked Write 7 Done Done Sent false
24881 7410 Acked Read 4 Done Done Sent false
24882 7411 Acked Read 4 Done Done Sent false
24883 7412 Acked Write 5 Done Done Sent false
24884 7413 Acked Read 8 Done Done Sent false
24885 7414 Acked Read 7 Done Done Sent false
24886 7415 Acked Write 8 Done Done Sent false
24887 7416 Acked Read 9 Done Done Sent false
24888 7417 Acked Write 10 Done Done Sent false
24889 7418 Acked Read 10 Done Done Sent false
24890 7419 Acked Read 3 Done Done Sent false
24891 7420 Acked Read 6 Done Done Sent false
24892 7421 Acked Read 8 Done Done Sent false
24893 7422 Acked Flush 0 Done Done Sent false
24894 7423 Acked Read 8 Done Done Sent false
24895 7424 Acked Read 3 Done Done Sent false
24896 7425 Acked Write 9 Done Done Sent false
24897 7426 Acked Write 5 Done Done Sent false
24898 7427 Acked Read 10 Done Done Sent false
24899 7428 Acked Read 2 Done Done Sent false
24900 7429 Acked Write 2 Done Done Sent false
24901 7430 Acked Write 5 Done Done Sent false
24902 7431 Acked Write 7 Done Done Sent false
24903 7432 Acked Write 8 Done Done Sent false
24904 7433 Acked Write 6 Done Done Sent false
24905 7434 Acked Read 7 Done Done Sent false
24906 7435 Acked Write 1 Done Done Sent false
24907 7436 Acked Write 8 Done Done Sent false
24908 7437 Acked Write 1 Done Done Sent false
24909 7438 Acked Read 9 Done Done Sent false
24910 7439 Acked Read 3 Done Done Sent false
24911 7440 Acked Read 1 Done Done Sent false
24912 7441 Acked Write 1 Done Done Sent false
24913 7442 Acked Write 4 Done Done Sent false
24914 7443 Acked Read 5 Done Done Sent false
24915 7444 Acked Write 9 Done Done Sent false
24916 7445 Acked Read 3 Done Done Sent false
24917 7446 Acked Read 9 Done Done Sent false
24918 7447 Acked Flush 0 Done Done Sent false
24919 7448 Acked Write 1 Done Done Sent false
24920 7449 Acked Read 8 Done Done Sent false
24921 7450 Acked Read 4 Done Done Sent false
24922 7451 Acked Read 3 Done Done Sent false
24923 7452 Acked Write 8 Done Done Sent false
24924 7453 Acked Read 1 Done Done Sent false
24925 7454 Acked Write 9 Done Done Sent false
24926 7455 Acked Write 10 Done Done Sent false
24927 7456 Acked Read 9 Done Done Sent false
24928 7457 Acked Read 7 Done Done Sent false
24929 7458 Acked Write 6 Done Done Sent false
24930 7459 Acked Read 3 Done Done Sent false
24931 7460 Acked Read 8 Done Done Sent false
24932 7461 Acked Read 5 Done Done Sent false
24933 7462 Acked Write 2 Done Done Sent false
24934 7463 Acked Read 1 Done Done Sent false
24935 7464 Acked Flush 0 Done Done Sent false
24936 7465 Acked Flush 0 Done Done Sent false
24937 7466 Acked Flush 0 Done Done Sent false
24938 7467 Acked Read 5 Done Done Sent false
24939 7468 Acked Write 9 Done Done Sent false
24940 7469 Acked Read 1 Done Done Sent false
24941 7470 Acked Write 5 Done Done Sent false
24942 7471 Acked Write 7 Done Done Sent false
24943 7472 Acked Read 7 Done Done Sent false
24944 7473 Acked Read 5 Done Done Sent false
24945 7474 Acked Write 3 Done Done Sent false
24946 7475 Acked Read 5 Done Done Sent false
24947 7476 Acked Read 10 Done Done Sent false
24948 7477 Acked Read 5 Done Done Sent false
24949 7478 Acked Read 6 Done Done Sent false
24950 7479 Acked Write 7 Done Done Sent false
24951 7480 Acked Flush 0 Done Done Sent false
24952 7481 Acked Write 8 Done Done Sent false
24953 7482 Acked Read 6 Done Done Sent false
24954 7483 Acked Write 3 Done Done Sent false
24955 7484 Acked Write 4 Done Done Sent false
24956 7485 Acked Write 4 Done Done Sent false
24957 7486 Acked Read 4 Done Done Sent false
24958 7487 Acked Write 3 Done Done Sent false
24959 7488 Acked Read 3 Done Done Sent false
24960 7489 Acked Read 4 Done Done Sent false
24961 7490 Acked Read 9 Done Done Sent false
24962 7491 Acked Read 4 Done Done Sent false
24963 7492 Acked Write 2 Done Done Sent false
24964 7493 Acked Write 10 Done Done Sent false
24965 7494 Acked Read 6 Done Done Sent false
24966 7495 Acked Write 10 Done Done Sent false
24967 7496 Acked Read 2 Done Done Sent false
24968 7497 Acked Flush 0 Done Done Sent false
24969 7498 Acked Read 4 Done Done Sent false
24970 7499 Acked Read 8 Done Done Sent false
24971 7500 Acked Read 1 Done Done Sent false
24972 7501 Acked Read 9 Done Done Sent false
24973 7502 Acked Read 10 Done Done Sent false
24974 7503 Acked Read 5 Done Done Sent false
24975 7504 Acked Read 8 Done Done Sent false
24976 7505 Acked Write 9 Done Done Sent false
24977 7506 Acked Write 10 Done Done Sent false
24978 7507 Acked Write 3 Done Done Sent false
24979 7508 Acked Write 1 Done Done Sent false
24980 7509 Acked Write 9 Done Done Sent false
24981 7510 Acked Write 10 Done Done Sent false
24982 7511 Acked Flush 0 Done Done Sent false
24983 7512 Acked Read 2 Done Done Sent false
24984 7513 Acked Read 9 Done Done Sent false
24985 7514 Acked Write 5 Done Done Sent false
24986 7515 Acked Write 4 Done Done Sent false
24987 7516 Acked Read 4 Done Done Sent false
24988 7517 Acked Read 3 Done Done Sent false
24989 7518 Acked Write 2 Done Done Sent false
24990 7519 Acked Write 4 Done Done Sent false
24991 7520 Acked Read 5 Done Done Sent false
24992 7521 Acked Read 7 Done Done Sent false
24993 7522 Acked Read 7 Done Done Sent false
24994 7523 Acked Write 8 Done Done Sent false
24995 7524 Acked Write 7 Done Done Sent false
24996 7525 Acked Write 7 Done Done Sent false
24997 7526 Acked Read 2 Done Done Sent false
24998 7527 Acked Read 6 Done Done Sent false
24999 7528 Acked Write 9 Done Done Sent false
25000 7529 Acked Read 2 Done Done Sent false
25001 7530 Acked Read 2 Done Done Sent false
25002 7531 Acked Write 2 Done Done Sent false
25003 7532 Acked Write 7 Done Done Sent false
25004 7533 Acked Read 6 Done Done Sent false
25005 7534 Acked Read 3 Done Done Sent false
25006 7535 Acked Write 8 Done Done Sent false
25007 7536 Acked Write 6 Done Done Sent false
25008 7537 Acked Flush 0 Done Done Sent false
25009 7538 Acked Write 5 Done Done Sent false
25010 7539 Acked Write 9 Done Done Sent false
25011 7540 Acked Read 9 Done Done Sent false
25012 7541 Acked Flush 0 Done Done Sent false
25013 7542 Acked Write 10 Done Done Sent false
25014 7543 Acked Write 7 Done Done Sent false
25015 7544 Acked Write 10 Done Done Sent false
25016 7545 Acked Read 9 Done Done Sent false
25017 7546 Acked Read 2 Done Done Sent false
25018 7547 Acked Flush 0 Done Done Sent false
25019 7548 Acked Read 1 Done Done Sent false
25020 7549 Acked Read 9 Done Done Sent false
25021 7550 Acked Write 4 Done Done Sent false
25022 7551 Acked Read 1 Done Done Sent false
25023 7552 Acked Read 4 Done Done Sent false
25024 7553 Acked Read 5 Done Done Sent false
25025 7554 Acked Flush 0 Done Done Sent false
25026 7555 Acked Read 3 Done Done Sent false
25027 7556 Acked Write 3 Done Done Sent false
25028 7557 Acked Read 4 Done Done Sent false
25029 7558 Acked Write 2 Done Done Sent false
25030 7559 Acked Read 4 Done Done Sent false
25031 7560 Acked Write 4 Done Done Sent false
25032 7561 Acked Read 1 Done Done Sent false
25033 7562 Acked Read 1 Done Done Sent false
25034 7563 Acked Write 1 Done Done Sent false
25035 7564 Acked Read 1 Done Done Sent false
25036 7565 Acked Read 10 Done Done Sent false
25037 7566 Acked Write 3 Done Done Sent false
25038 7567 Acked Write 6 Done Done Sent false
25039 7568 Acked Read 5 Done Done Sent false
25040 7569 Acked Read 6 Done Done Sent false
25041 7570 Acked Read 4 Done Done Sent false
25042 7571 Acked Flush 0 Done Done Sent false
25043 7572 Acked Read 8 Done Done Sent false
25044 7573 Acked Write 6 Done Done Sent false
25045 7574 Acked Read 6 Done Done Sent false
25046 7575 Acked Read 9 Done Done Sent false
25047 7576 Acked Write 8 Done Done Sent false
25048 7577 Acked Write 3 Done Done Sent false
25049 7578 Acked Write 10 Done Done Sent false
25050 7579 Acked Write 1 Done Done Sent false
25051 7580 Acked Read 5 Done Done Sent false
25052 7581 Acked Flush 0 Done Done Sent false
25053 7582 Acked Write 1 Done Done Sent false
25054 7583 Acked Write 6 Done Done Sent false
25055 7584 Acked Read 8 Done Done Sent false
25056 7585 Acked Read 7 Done Done Sent false
25057 7586 Acked Write 10 Done Done Sent false
25058 7587 Acked Read 10 Done Done Sent false
25059 7588 Acked Flush 0 Done Done Sent false
25060 7589 Acked Write 8 Done Done Sent false
25061 7590 Acked Read 4 Done Done Sent false
25062 7591 Acked Read 5 Done Done Sent false
25063 7592 Acked Read 4 Done Done Sent false
25064 7593 Acked Write 10 Done Done Sent false
25065 7594 Acked Write 8 Done Done Sent false
25066 7595 Acked Write 10 Done Done Sent false
25067 7596 Acked Read 10 Done Done Sent false
25068 7597 Acked Flush 0 Done Done Sent false
25069 7598 Acked Read 9 Done Done Sent false
25070 7599 Acked Write 8 Done Done Sent false
25071 7600 Acked Read 10 Done Done Sent false
25072 7601 Acked Read 4 Done Done Sent false
25073 7602 Acked Read 7 Done Done Sent false
25074 7603 Acked Write 4 Done Done Sent false
25075 7604 Acked Write 3 Done Done Sent false
25076 7605 Acked Write 3 Done Done Sent false
25077 7606 Acked Write 10 Done Done Sent false
25078 7607 Acked Write 5 Done Done Sent false
25079 7608 Acked Flush 0 Done Done Sent false
25080 7609 Acked Read 1 Done Done Sent false
25081 7610 Acked Read 7 Done Done Sent false
25082 7611 Acked Read 9 Done Done Sent false
25083 7612 Acked Write 7 Done Done Sent false
25084 7613 Acked Read 8 Done Done Sent false
25085 7614 Acked Read 10 Done Done Sent false
25086 7615 Acked Read 8 Done Done Sent false
25087 7616 Acked Read 3 Done Done Sent false
25088 7617 Acked Read 9 Done Done Sent false
25089 7618 Acked Read 6 Done Done Sent false
25090 7619 Acked Read 3 Done Done Sent false
25091 7620 Acked Write 6 Done Done Sent false
25092 7621 Acked Write 6 Done Done Sent false
25093 7622 Acked Write 1 Done Done Sent false
25094 7623 Acked Read 8 Done Done Sent false
25095 7624 Acked Read 4 Done Done Sent false
25096 7625 Acked Flush 0 Done Done Sent false
25097 7626 Acked Write 8 Done Done Sent false
25098 7627 Acked Flush 0 Done Done Sent false
25099 7628 Acked Write 2 Done Done Sent false
25100 7629 Acked Read 6 Done Done Sent false
25101 7630 Acked Read 2 Done Done Sent false
25102 7631 Acked Read 4 Done Done Sent false
25103 7632 Acked Read 8 Done Done Sent false
25104 7633 Acked Read 9 Done Done Sent false
25105 7634 Acked Read 2 Done Done Sent false
25106 7635 Acked Write 7 Done Done Sent false
25107 7636 Acked Read 8 Done Done Sent false
25108 7637 Acked Read 2 Done Done Sent false
25109 7638 Acked Flush 0 Done Done Sent false
25110 7639 Acked Write 10 Done Done Sent false
25111 7640 Acked Write 8 Done Done Sent false
25112 7641 Acked Read 4 Done Done Sent false
25113 7642 Acked Write 3 Done Done Sent false
25114 7643 Acked Read 5 Done Done Sent false
25115 7644 Acked Flush 0 Done Done Sent false
25116 7645 Acked Read 3 Done Done Sent false
25117 7646 Acked Flush 0 Done Done Sent false
25118 7647 Acked Write 5 Done Done Sent false
25119 7648 Acked Write 9 Done Done Sent false
25120 7649 Acked Write 8 Done Done Sent false
25121 7650 Acked Write 7 Done Done Sent false
25122 STATES DS:0 DS:1 DS:2 TOTAL
25123 Sent 0 0 301 301
25124 Done 301 301 0 602
25125 Skipped 0 0 0 0
25126 Error 0 0 0 0
25127 Last Flush: 7646 7646 7349
25128 Downstairs last five completed: 7349 7348 7347 7346 7345
25129 Upstairs last five completed: 7650 7649 7648 7647 7646
25130 CLIENT: Up:0 ds:301 act:2
25131 [8830] got start action
25132 [8830] I am going to restart
25133 Starting downstairs at port 8830
25134 Make output file at "/tmp/dsc/downstairs-8830.txt"
25135 [8830][2] reports Starting
25136 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1221)
25137 [8830][2] reports Running
25138 ----------------------------------------------------------------
25139 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25140 States: Active Active Offline
25141 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25142 7350 Acked Flush 0 Done Done Sent false
25143 7351 Acked Write 6 Done Done Sent false
25144 7352 Acked Read 5 Done Done Sent false
25145 7353 Acked Read 1 Done Done Sent false
25146 7354 Acked Read 6 Done Done Sent false
25147 7355 Acked Read 4 Done Done Sent false
25148 7356 Acked Write 2 Done Done Sent false
25149 7357 Acked Write 7 Done Done Sent false
25150 7358 Acked Write 5 Done Done Sent false
25151 7359 Acked Read 5 Done Done Sent false
25152 7360 Acked Write 2 Done Done Sent false
25153 7361 Acked Read 9 Done Done Sent false
25154 7362 Acked Write 4 Done Done Sent false
25155 7363 Acked Read 6 Done Done Sent false
25156 7364 Acked Write 4 Done Done Sent false
25157 7365 Acked Read 10 Done Done Sent false
25158 7366 Acked Read 1 Done Done Sent false
25159 7367 Acked Read 8 Done Done Sent false
25160 7368 Acked Flush 0 Done Done Sent false
25161 7369 Acked Write 10 Done Done Sent false
25162 7370 Acked Read 8 Done Done Sent false
25163 7371 Acked Write 7 Done Done Sent false
25164 7372 Acked Write 5 Done Done Sent false
25165 7373 Acked Read 2 Done Done Sent false
25166 7374 Acked Read 7 Done Done Sent false
25167 7375 Acked Write 6 Done Done Sent false
25168 7376 Acked Read 2 Done Done Sent false
25169 7377 Acked Read 8 Done Done Sent false
25170 7378 Acked Write 3 Done Done Sent false
25171 7379 Acked Read 7 Done Done Sent false
25172 7380 Acked Read 5 Done Done Sent false
25173 7381 Acked Write 8 Done Done Sent false
25174 7382 Acked Write 5 Done Done Sent false
25175 7383 Acked Read 9 Done Done Sent false
25176 7384 Acked Read 4 Done Done Sent false
25177 7385 Acked Flush 0 Done Done Sent false
25178 7386 Acked Write 2 Done Done Sent false
25179 7387 Acked Write 6 Done Done Sent false
25180 7388 Acked Write 10 Done Done Sent false
25181 7389 Acked Read 10 Done Done Sent false
25182 7390 Acked Write 4 Done Done Sent false
25183 7391 Acked Read 3 Done Done Sent false
25184 7392 Acked Read 1 Done Done Sent false
25185 7393 Acked Flush 0 Done Done Sent false
25186 7394 Acked Write 2 Done Done Sent false
25187 7395 Acked Write 5 Done Done Sent false
25188 7396 Acked Read 4 Done Done Sent false
25189 7397 Acked Read 8 Done Done Sent false
25190 7398 Acked Read 3 Done Done Sent false
25191 7399 Acked Read 10 Done Done Sent false
25192 7400 Acked Read 8 Done Done Sent false
25193 7401 Acked Read 10 Done Done Sent false
25194 7402 Acked Read 3 Done Done Sent false
25195 7403 Acked Read 6 Done Done Sent false
25196 7404 Acked Write 8 Done Done Sent false
25197 7405 Acked Write 8 Done Done Sent false
25198 7406 Acked Write 4 Done Done Sent false
25199 7407 Acked Read 10 Done Done Sent false
25200 7408 Acked Write 1 Done Done Sent false
25201 7409 Acked Write 7 Done Done Sent false
25202 7410 Acked Read 4 Done Done Sent false
25203 7411 Acked Read 4 Done Done Sent false
25204 7412 Acked Write 5 Done Done Sent false
25205 7413 Acked Read 8 Done Done Sent false
25206 7414 Acked Read 7 Done Done Sent false
25207 7415 Acked Write 8 Done Done Sent false
25208 7416 Acked Read 9 Done Done Sent false
25209 7417 Acked Write 10 Done Done Sent false
25210 7418 Acked Read 10 Done Done Sent false
25211 7419 Acked Read 3 Done Done Sent false
25212 7420 Acked Read 6 Done Done Sent false
25213 7421 Acked Read 8 Done Done Sent false
25214 7422 Acked Flush 0 Done Done Sent false
25215 7423 Acked Read 8 Done Done Sent false
25216 7424 Acked Read 3 Done Done Sent false
25217 7425 Acked Write 9 Done Done Sent false
25218 7426 Acked Write 5 Done Done Sent false
25219 7427 Acked Read 10 Done Done Sent false
25220 7428 Acked Read 2 Done Done Sent false
25221 7429 Acked Write 2 Done Done Sent false
25222 7430 Acked Write 5 Done Done Sent false
25223 7431 Acked Write 7 Done Done Sent false
25224 7432 Acked Write 8 Done Done Sent false
25225 7433 Acked Write 6 Done Done Sent false
25226 7434 Acked Read 7 Done Done Sent false
25227 7435 Acked Write 1 Done Done Sent false
25228 7436 Acked Write 8 Done Done Sent false
25229 7437 Acked Write 1 Done Done Sent false
25230 7438 Acked Read 9 Done Done Sent false
25231 7439 Acked Read 3 Done Done Sent false
25232 7440 Acked Read 1 Done Done Sent false
25233 7441 Acked Write 1 Done Done Sent false
25234 7442 Acked Write 4 Done Done Sent false
25235 7443 Acked Read 5 Done Done Sent false
25236 7444 Acked Write 9 Done Done Sent false
25237 7445 Acked Read 3 Done Done Sent false
25238 7446 Acked Read 9 Done Done Sent false
25239 7447 Acked Flush 0 Done Done Sent false
25240 7448 Acked Write 1 Done Done Sent false
25241 7449 Acked Read 8 Done Done Sent false
25242 7450 Acked Read 4 Done Done Sent false
25243 7451 Acked Read 3 Done Done Sent false
25244 7452 Acked Write 8 Done Done Sent false
25245 7453 Acked Read 1 Done Done Sent false
25246 7454 Acked Write 9 Done Done Sent false
25247 7455 Acked Write 10 Done Done Sent false
25248 7456 Acked Read 9 Done Done Sent false
25249 7457 Acked Read 7 Done Done Sent false
25250 7458 Acked Write 6 Done Done Sent false
25251 7459 Acked Read 3 Done Done Sent false
25252 7460 Acked Read 8 Done Done Sent false
25253 7461 Acked Read 5 Done Done Sent false
25254 7462 Acked Write 2 Done Done Sent false
25255 7463 Acked Read 1 Done Done Sent false
25256 7464 Acked Flush 0 Done Done Sent false
25257 7465 Acked Flush 0 Done Done Sent false
25258 7466 Acked Flush 0 Done Done Sent false
25259 7467 Acked Read 5 Done Done Sent false
25260 7468 Acked Write 9 Done Done Sent false
25261 7469 Acked Read 1 Done Done Sent false
25262 7470 Acked Write 5 Done Done Sent false
25263 7471 Acked Write 7 Done Done Sent false
25264 7472 Acked Read 7 Done Done Sent false
25265 7473 Acked Read 5 Done Done Sent false
25266 7474 Acked Write 3 Done Done Sent false
25267 7475 Acked Read 5 Done Done Sent false
25268 7476 Acked Read 10 Done Done Sent false
25269 7477 Acked Read 5 Done Done Sent false
25270 7478 Acked Read 6 Done Done Sent false
25271 7479 Acked Write 7 Done Done Sent false
25272 7480 Acked Flush 0 Done Done Sent false
25273 7481 Acked Write 8 Done Done Sent false
25274 7482 Acked Read 6 Done Done Sent false
25275 7483 Acked Write 3 Done Done Sent false
25276 7484 Acked Write 4 Done Done Sent false
25277 7485 Acked Write 4 Done Done Sent false
25278 7486 Acked Read 4 Done Done Sent false
25279 7487 Acked Write 3 Done Done Sent false
25280 7488 Acked Read 3 Done Done Sent false
25281 7489 Acked Read 4 Done Done Sent false
25282 7490 Acked Read 9 Done Done Sent false
25283 7491 Acked Read 4 Done Done Sent false
25284 7492 Acked Write 2 Done Done Sent false
25285 7493 Acked Write 10 Done Done Sent false
25286 7494 Acked Read 6 Done Done Sent false
25287 7495 Acked Write 10 Done Done Sent false
25288 7496 Acked Read 2 Done Done Sent false
25289 7497 Acked Flush 0 Done Done Sent false
25290 7498 Acked Read 4 Done Done Sent false
25291 7499 Acked Read 8 Done Done Sent false
25292 7500 Acked Read 1 Done Done Sent false
25293 7501 Acked Read 9 Done Done Sent false
25294 7502 Acked Read 10 Done Done Sent false
25295 7503 Acked Read 5 Done Done Sent false
25296 7504 Acked Read 8 Done Done Sent false
25297 7505 Acked Write 9 Done Done Sent false
25298 7506 Acked Write 10 Done Done Sent false
25299 7507 Acked Write 3 Done Done Sent false
25300 7508 Acked Write 1 Done Done Sent false
25301 7509 Acked Write 9 Done Done Sent false
25302 7510 Acked Write 10 Done Done Sent false
25303 7511 Acked Flush 0 Done Done Sent false
25304 7512 Acked Read 2 Done Done Sent false
25305 7513 Acked Read 9 Done Done Sent false
25306 7514 Acked Write 5 Done Done Sent false
25307 7515 Acked Write 4 Done Done Sent false
25308 7516 Acked Read 4 Done Done Sent false
25309 7517 Acked Read 3 Done Done Sent false
25310 7518 Acked Write 2 Done Done Sent false
25311 7519 Acked Write 4 Done Done Sent false
25312 7520 Acked Read 5 Done Done Sent false
25313 7521 Acked Read 7 Done Done Sent false
25314 7522 Acked Read 7 Done Done Sent false
25315 7523 Acked Write 8 Done Done Sent false
25316 7524 Acked Write 7 Done Done Sent false
25317 7525 Acked Write 7 Done Done Sent false
25318 7526 Acked Read 2 Done Done Sent false
25319 7527 Acked Read 6 Done Done Sent false
25320 7528 Acked Write 9 Done Done Sent false
25321 7529 Acked Read 2 Done Done Sent false
25322 7530 Acked Read 2 Done Done Sent false
25323 7531 Acked Write 2 Done Done Sent false
25324 7532 Acked Write 7 Done Done Sent false
25325 7533 Acked Read 6 Done Done Sent false
25326 7534 Acked Read 3 Done Done Sent false
25327 7535 Acked Write 8 Done Done Sent false
25328 7536 Acked Write 6 Done Done Sent false
25329 7537 Acked Flush 0 Done Done Sent false
25330 7538 Acked Write 5 Done Done Sent false
25331 7539 Acked Write 9 Done Done Sent false
25332 7540 Acked Read 9 Done Done Sent false
25333 7541 Acked Flush 0 Done Done Sent false
25334 7542 Acked Write 10 Done Done Sent false
25335 7543 Acked Write 7 Done Done Sent false
25336 7544 Acked Write 10 Done Done Sent false
25337 7545 Acked Read 9 Done Done Sent false
25338 7546 Acked Read 2 Done Done Sent false
25339 7547 Acked Flush 0 Done Done Sent false
25340 7548 Acked Read 1 Done Done Sent false
25341 7549 Acked Read 9 Done Done Sent false
25342 7550 Acked Write 4 Done Done Sent false
25343 7551 Acked Read 1 Done Done Sent false
25344 7552 Acked Read 4 Done Done Sent false
25345 7553 Acked Read 5 Done Done Sent false
25346 7554 Acked Flush 0 Done Done Sent false
25347 7555 Acked Read 3 Done Done Sent false
25348 7556 Acked Write 3 Done Done Sent false
25349 7557 Acked Read 4 Done Done Sent false
25350 7558 Acked Write 2 Done Done Sent false
25351 7559 Acked Read 4 Done Done Sent false
25352 7560 Acked Write 4 Done Done Sent false
25353 7561 Acked Read 1 Done Done Sent false
25354 7562 Acked Read 1 Done Done Sent false
25355 7563 Acked Write 1 Done Done Sent false
25356 7564 Acked Read 1 Done Done Sent false
25357 7565 Acked Read 10 Done Done Sent false
25358 7566 Acked Write 3 Done Done Sent false
25359 7567 Acked Write 6 Done Done Sent false
25360 7568 Acked Read 5 Done Done Sent false
25361 7569 Acked Read 6 Done Done Sent false
25362 7570 Acked Read 4 Done Done Sent false
25363 7571 Acked Flush 0 Done Done Sent false
25364 7572 Acked Read 8 Done Done Sent false
25365 7573 Acked Write 6 Done Done Sent false
25366 7574 Acked Read 6 Done Done Sent false
25367 7575 Acked Read 9 Done Done Sent false
25368 7576 Acked Write 8 Done Done Sent false
25369 7577 Acked Write 3 Done Done Sent false
25370 7578 Acked Write 10 Done Done Sent false
25371 7579 Acked Write 1 Done Done Sent false
25372 7580 Acked Read 5 Done Done Sent false
25373 7581 Acked Flush 0 Done Done Sent false
25374 7582 Acked Write 1 Done Done Sent false
25375 7583 Acked Write 6 Done Done Sent false
25376 7584 Acked Read 8 Done Done Sent false
25377 7585 Acked Read 7 Done Done Sent false
25378 7586 Acked Write 10 Done Done Sent false
25379 7587 Acked Read 10 Done Done Sent false
25380 7588 Acked Flush 0 Done Done Sent false
25381 7589 Acked Write 8 Done Done Sent false
25382 7590 Acked Read 4 Done Done Sent false
25383 7591 Acked Read 5 Done Done Sent false
25384 7592 Acked Read 4 Done Done Sent false
25385 7593 Acked Write 10 Done Done Sent false
25386 7594 Acked Write 8 Done Done Sent false
25387 7595 Acked Write 10 Done Done Sent false
25388 7596 Acked Read 10 Done Done Sent false
25389 7597 Acked Flush 0 Done Done Sent false
25390 7598 Acked Read 9 Done Done Sent false
25391 7599 Acked Write 8 Done Done Sent false
25392 7600 Acked Read 10 Done Done Sent false
25393 7601 Acked Read 4 Done Done Sent false
25394 7602 Acked Read 7 Done Done Sent false
25395 7603 Acked Write 4 Done Done Sent false
25396 7604 Acked Write 3 Done Done Sent false
25397 7605 Acked Write 3 Done Done Sent false
25398 7606 Acked Write 10 Done Done Sent false
25399 7607 Acked Write 5 Done Done Sent false
25400 7608 Acked Flush 0 Done Done Sent false
25401 7609 Acked Read 1 Done Done Sent false
25402 7610 Acked Read 7 Done Done Sent false
25403 7611 Acked Read 9 Done Done Sent false
25404 7612 Acked Write 7 Done Done Sent false
25405 7613 Acked Read 8 Done Done Sent false
25406 7614 Acked Read 10 Done Done Sent false
25407 7615 Acked Read 8 Done Done Sent false
25408 7616 Acked Read 3 Done Done Sent false
25409 7617 Acked Read 9 Done Done Sent false
25410 7618 Acked Read 6 Done Done Sent false
25411 7619 Acked Read 3 Done Done Sent false
25412 7620 Acked Write 6 Done Done Sent false
25413 7621 Acked Write 6 Done Done Sent false
25414 7622 Acked Write 1 Done Done Sent false
25415 7623 Acked Read 8 Done Done Sent false
25416 7624 Acked Read 4 Done Done Sent false
25417 7625 Acked Flush 0 Done Done Sent false
25418 7626 Acked Write 8 Done Done Sent false
25419 7627 Acked Flush 0 Done Done Sent false
25420 7628 Acked Write 2 Done Done Sent false
25421 7629 Acked Read 6 Done Done Sent false
25422 7630 Acked Read 2 Done Done Sent false
25423 7631 Acked Read 4 Done Done Sent false
25424 7632 Acked Read 8 Done Done Sent false
25425 7633 Acked Read 9 Done Done Sent false
25426 7634 Acked Read 2 Done Done Sent false
25427 7635 Acked Write 7 Done Done Sent false
25428 7636 Acked Read 8 Done Done Sent false
25429 7637 Acked Read 2 Done Done Sent false
25430 7638 Acked Flush 0 Done Done Sent false
25431 7639 Acked Write 10 Done Done Sent false
25432 7640 Acked Write 8 Done Done Sent false
25433 7641 Acked Read 4 Done Done Sent false
25434 7642 Acked Write 3 Done Done Sent false
25435 7643 Acked Read 5 Done Done Sent false
25436 7644 Acked Flush 0 Done Done Sent false
25437 7645 Acked Read 3 Done Done Sent false
25438 7646 Acked Flush 0 Done Done Sent false
25439 7647 Acked Write 5 Done Done Sent false
25440 7648 Acked Write 9 Done Done Sent false
25441 7649 Acked Write 8 Done Done Sent false
25442 7650 Acked Write 7 Done Done Sent false
25443 7651 Acked Flush 0 Done Done Sent false
25444 STATES DS:0 DS:1 DS:2 TOTAL
25445 Sent 0 0 302 302
25446 Done 302 302 0 604
25447 Skipped 0 0 0 0
25448 Error 0 0 0 0
25449 Last Flush: 7651 7651 7349
25450 Downstairs last five completed: 7349 7348 7347 7346 7345
25451 Upstairs last five completed: 7651 7650 7649 7648 7647
25452 CLIENT: Up:0 ds:302 act:2
254532024-11-07T14:08:38.015ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254542024-11-07T14:08:38.015ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51199) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254552024-11-07T14:08:38.016ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254562024-11-07T14:08:38.016ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254572024-11-07T14:08:38.016ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254582024-11-07T14:08:38.016ZINFOcrucible: send last flush ID to this DS: 7349 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254592024-11-07T14:08:38.017ZINFOcrucible: Replied this last flush ID: 7349 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254602024-11-07T14:08:38.017ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254612024-11-07T14:08:38.017ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254622024-11-07T14:08:38.017ZINFOcrucible: [2] client re-new 302 jobs since flush 7349 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254632024-11-07T14:08:38.017ZINFOcrucible: [2] Replayed 302 jobs since flush: 7349 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
25464 ----------------------------------------------------------------
25465 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25466 Downstairs last five completed: 7651 7650 7649 7648 7647
25467 Upstairs last five completed: 7651 7650 7649 7648 7647
25468 CLIENT: Up:0 ds:0 act:3
25469 Replay: All jobs finished
25470 Main task has work to do, go find it
25471 got dsc Stop(2)
25472 stop 2
25473 [8830] Got stop action so:false kr:false
25474 Nov 07 14:08:40.318 INFO request completed, latency_us: 350, response_code: 204, uri: /stop/cid/2, method: POST, req_id: d84e325e-3692-4c61-a96e-0151828843b4, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
254752024-11-07T14:08:40.318ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254762024-11-07T14:08:40.318ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254772024-11-07T14:08:40.318ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254782024-11-07T14:08:40.318ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
254792024-11-07T14:08:40.318ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
25480 Nov 07 14:08:40.319 INFO request completed, latency_us: 126, response_code: 200, uri: /state/cid/2, method: GET, req_id: ea611246-cabd-4160-a92a-fe08f6cc1eaf{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:08:40.319062603Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"":"io task","client":"2","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
25481 , remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
25482 [8830] Exited with: ExitStatus(unix_wait_status(9))
25483 [8830][2] reports Exit
25484 Nov 07 14:08:44.321 INFO request completed, latency_us: 221, response_code: 200, uri: /state/cid/2, method: GET, req_id: 84c61ff5-93a9-469c-91b5-827d5cfe2217, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
25485 001/300 Read block 1594 len 36864
25486 002/300 Read block 4633 len 16384
25487 003/300 Read block 1758 len 40960
25488 004/300 Flush
25489 005/300 Read block 2805 len 8192
25490 006/300 Write block 1485 len 8192 data: 8 7
25491 007/300 Write block 1804 len 36864 data: 5 6 6 6 6 5 5 5 4
25492 008/300 Read block 636 len 8192
25493 009/300 Read block 185 len 32768
25494 010/300 Write block 405 len 32768 data: 9 8 8 8 7 9 9 6
25495 011/300 Read block 752 len 8192
25496 012/300 Write block 351 len 8192 data: 3 3
25497 013/300 Write block 440 len 16384 data: 8 6 7 6
25498 014/300 Write block 3339 len 36864 data: 6 5 5 6 5 6 7 6 5
25499 015/300 Write block 4443 len 28672 data: 7 7 6 5 5 5 5
25500 016/300 Flush
25501 017/300 Flush
25502 018/300 Write block 3399 len 36864 data: 5 6 7 8 8 8 6 6 4
25503 019/300 Read block 1756 len 20480
25504 020/300 Write block 1848 len 16384 data: 4 4 6 5
25505 021/300 Read block 1521 len 4096
25506 022/300 Read block 1089 len 32768
25507 023/300 Read block 26 len 20480
25508 024/300 Read block 1965 len 12288
25509 025/300 Write block 2754 len 40960 data: 5 5 5 5 4 3 2 3 4 3
25510 026/300 Write block 811 len 20480 data: 5 3 3 2 2
25511 027/300 Write block 2552 len 40960 data: 5 3 5 5 5 5 4 4 4 3
25512 028/300 Read block 1882 len 28672
25513 029/300 Read block 4663 len 36864
25514 030/300 Write block 3148 len 8192 data: 4 4
25515 031/300 Read block 4911 len 28672
25516 032/300 Write block 4411 len 28672 data: 3 3 4 4 3 4 6
25517 033/300 Read block 3399 len 20480
25518 034/300 Write block 4655 len 8192 data: 3 3
25519 035/300 Read block 3366 len 28672
25520 036/300 Write block 3654 len 36864 data: 5 4 4 5 4 4 5 6 5
25521 037/300 Write block 2849 len 12288 data: 6 6 6
25522 038/300 Read block 1092 len 24576
25523 039/300 Write block 601 len 16384 data: 3 3 3 4
25524 040/300 Write block 2474 len 12288 data: 6 7 7
25525 041/300 Read block 2085 len 4096
25526 042/300 Read block 42 len 28672
25527 043/300 Write block 2348 len 36864 data: 3 3 3 3 3 3 3 4 4
25528 044/300 Flush
25529 045/300 Read block 2761 len 12288
25530 046/300 Read block 2905 len 24576
25531 047/300 Flush
25532 048/300 Write block 2964 len 40960 data: 6 5 4 4 3 5 5 6 6 6
25533 049/300 Write block 3567 len 12288 data: 5 5 6
25534 050/300 Flush
25535 051/300 Read block 3452 len 12288
25536 052/300 Read block 4403 len 24576
25537 053/300 Write block 102 len 8192 data: 2 2
25538 054/300 Write block 1617 len 40960 data: 4 3 3 4 4 3 2 3 3 4
25539 055/300 Write block 3750 len 36864 data: 6 7 7 8 7 5 4 4 4
25540 056/300 Write block 4177 len 32768 data: 6 7 6 5 5 5 5 4
25541 057/300 Read block 3458 len 32768
25542 058/300 Write block 4669 len 12288 data: 6 5 6
25543 059/300 Read block 1986 len 4096
25544 060/300 Read block 3795 len 8192
25545 061/300 Write block 3545 len 32768 data: 8 8 7 9 8 7 8 10
25546 062/300 Flush
25547 063/300 Read block 3269 len 24576
25548 064/300 Write block 2679 len 32768 data: 2 2 2 3 4 4 4 4
25549 065/300 Write block 1826 len 20480 data: 5 6 8 6 7
25550 066/300 Write block 1548 len 28672 data: 4 3 3 2 2 3 5
25551 067/300 Write block 4228 len 8192 data: 8 8
25552 068/300 Read block 1331 len 28672
25553 069/300 Write block 1876 len 28672 data: 7 8 7 7 5 5 5
25554 070/300 Read block 3688 len 8192
25555 071/300 Read block 126 len 24576
25556 072/300 Write block 545 len 24576 data: 6 5 6 5 4 6
25557 073/300 Read block 1683 len 40960
25558 074/300 Read block 4388 len 40960
25559 075/300 Read block 4392 len 28672
25560 076/300 Read block 2136 len 12288
25561 077/300 Flush
25562 078/300 Read block 4523 len 40960
25563 079/300 Flush
25564 080/300 Read block 4043 len 24576
25565 081/300 Read block 2496 len 4096
25566 082/300 Read block 2533 len 36864
25567 083/300 Read block 4408 len 32768
25568 084/300 Read block 3862 len 4096
25569 085/300 Write block 4707 len 16384 data: 3 3 3 4
25570 086/300 Write block 3536 len 4096 data: 5
25571 087/300 Write block 4035 len 12288 data: 5 5 5
25572 088/300 Write block 2007 len 20480 data: 7 8 7 7 7
25573 089/300 Read block 1455 len 40960
25574 090/300 Read block 4393 len 40960
25575 091/300 Write block 855 len 24576 data: 6 6 6 5 6 7
25576 092/300 Flush
25577 093/300 Flush
25578 094/300 Read block 3924 len 32768
25579 095/300 Write block 1074 len 36864 data: 4 4 4 4 5 5 6 6 6
25580 096/300 Read block 4805 len 28672
25581 097/300 Write block 3239 len 8192 data: 5 9
25582 098/300 Read block 1819 len 24576
25583 099/300 Read block 2791 len 16384
25584 100/300 Read block 1285 len 28672
25585 101/300 Flush
25586 102/300 Write block 1938 len 36864 data: 4 3 5 5 6 8 7 7 8
25587 103/300 Write block 3726 len 32768 data: 5 4 3 2 4 7 7 6
25588 104/300 Read block 3720 len 28672
25589 105/300 Flush
25590 106/300 Write block 2415 len 12288 data: 4 4 5
25591 107/300 Flush
25592 108/300 Write block 1288 len 8192 data: 4 6
25593 109/300 Write block 3767 len 36864 data: 3 2 2 2 2 2 2 2 2
25594 110/300 Read block 1526 len 28672
25595 111/300 Write block 511 len 32768 data: 4 5 5 6 7 6 4 5
25596 112/300 Flush
25597 113/300 Read block 2939 len 12288
25598 114/300 Read block 3274 len 32768
25599 115/300 Read block 3930 len 28672
25600 116/300 Write block 705 len 40960 data: 5 8 10 9 10 8 8 8 8 6
25601 117/300 Read block 1599 len 28672
25602 118/300 Write block 2312 len 28672 data: 5 3 3 3 4 4 5
25603 119/300 Write block 3632 len 4096 data: 8
25604 120/300 Flush
25605 121/300 Write block 4646 len 40960 data: 6 8 8 6 4 4 3 2 2 4
25606 122/300 Write block 4704 len 12288 data: 3 3 3
25607 123/300 Write block 2724 len 20480 data: 6 6 6 5 5
25608 124/300 Write block 2575 len 24576 data: 3 4 4 5 5 5
25609 125/300 Write block 603 len 36864 data: 4 5 5 5 4 3 4 5 5
25610 126/300 Read block 3386 len 28672
25611 127/300 Read block 1527 len 32768
25612 128/300 Write block 4975 len 32768 data: 3 3 3 3 3 4 4 4
25613 129/300 Write block 827 len 40960 data: 3 5 6 6 5 5 5 5 4 4
25614 130/300 Read block 3431 len 24576
25615 131/300 Flush
25616 132/300 Read block 2728 len 8192
25617 133/300 Write block 4990 len 40960 data: 5 5 4 2 3 3 3 3 3 2
25618 134/300 Read block 4518 len 4096
25619 135/300 Read block 1490 len 40960
25620 136/300 Flush
25621 137/300 Read block 872 len 8192
25622 138/300 Read block 131 len 16384
25623 139/300 Read block 2624 len 4096
25624 140/300 Write block 1433 len 16384 data: 6 5 4 5
25625 141/300 Write block 1900 len 16384 data: 4 5 5 6
25626 142/300 Write block 1594 len 16384 data: 5 4 5 7
25627 143/300 Read block 2612 len 32768
25628 144/300 Write block 1108 len 8192 data: 5 5
25629 145/300 Write block 2281 len 4096 data: 4
25630 146/300 Write block 2538 len 24576 data: 6 10 10 11 10 10
25631 147/300 Write block 2135 len 16384 data: 4 2 3 4
25632 148/300 Read block 4274 len 8192
25633 149/300 Read block 3349 len 20480
25634 150/300 Read block 1297 len 20480
25635 151/300 Flush
25636 152/300 Read block 2916 len 8192
25637 153/300 Write block 4672 len 20480 data: 7 6 6 6 5
25638 154/300 Write block 3050 len 36864 data: 6 6 6 6 7 4 5 3 4
25639 155/300 Write block 4958 len 28672 data: 9 9 9 6 6 5 5
25640 156/300 Flush
25641 157/300 Read block 1190 len 28672
25642 158/300 Read block 322 len 36864
25643 159/300 Write block 2729 len 36864 data: 4 5 6 7 6 5 3 3 4
25644 160/300 Flush
25645 161/300 Read block 1723 len 16384
25646 162/300 Read block 4208 len 20480
25647 163/300 Read block 4887 len 32768
25648 164/300 Read block 2253 len 36864
25649 165/300 Read block 617 len 16384
25650 166/300 Read block 450 len 12288
25651 167/300 Write block 2350 len 40960 data: 4 4 4 4 4 5 5 5 5 5
25652 168/300 Read block 2279 len 24576
25653 169/300 Read block 4210 len 16384
25654 170/300 Flush
25655 171/300 Read block 2458 len 20480
25656 172/300 Read block 1549 len 12288
25657 173/300 Read block 4554 len 40960
25658 174/300 Flush
25659 175/300 Read block 2843 len 8192
25660 176/300 Read block 1707 len 32768
25661 177/300 Read block 1981 len 24576
25662 178/300 Write block 4792 len 36864 data: 6 5 6 7 7 7 7 7 6
25663 179/300 Write block 3822 len 16384 data: 6 6 5 4
25664 180/300 Read block 4110 len 8192
25665 181/300 Write block 3502 len 12288 data: 4 5 5
25666 182/300 Flush
25667 183/300 Read block 1795 len 32768
25668 184/300 Read block 3603 len 12288
25669 185/300 Write block 1294 len 36864 data: 2 2 2 3 3 4 4 4 4
25670 186/300 Write block 1051 len 28672 data: 8 7 6 5 5 4 2
25671 187/300 Read block 1674 len 24576
25672 188/300 Read block 261 len 32768
25673 189/300 Read block 2425 len 8192
25674 190/300 Write block 805 len 24576 data: 5 5 4 4 4 5
25675 191/300 Read block 3428 len 20480
25676 192/300 Write block 4597 len 20480 data: 6 5 5 5 5
25677 193/300 Write block 3094 len 40960 data: 3 3 3 2 2 3 3 3 4 4
25678 194/300 Write block 1748 len 32768 data: 3 5 8 9 8 7 7 6
25679 195/300 Read block 2950 len 28672
25680 196/300 Read block 3834 len 4096
25681 197/300 Read block 758 len 16384
25682 198/300 Write block 4701 len 36864 data: 7 6 4 4 4 4 4 4 4
25683 199/300 Read block 998 len 12288
25684 200/300 Write block 260 len 32768 data: 5 7 7 8 6 7 6 5
25685 201/300 Read block 2213 len 4096
25686 202/300 Read block 4039 len 32768
25687 203/300 Write block 108 len 12288 data: 2 2 2
25688 204/300 Read block 3323 len 4096
25689 205/300 Read block 2975 len 36864
25690 206/300 Write block 3590 len 16384 data: 7 6 4 3
25691 207/300 Read block 1731 len 28672
25692 208/300 Read block 163 len 12288
25693 209/300 Read block 4219 len 16384
25694 210/300 Read block 969 len 12288
25695 211/300 Write block 195 len 4096 data: 5
25696 212/300 Write block 3935 len 40960 data: 4 2 3 4 3 3 2 2 2 2
25697 213/300 Write block 1019 len 36864 data: 3 3 2 3 4 4 4 4 4
25698 214/300 Read block 953 len 36864
25699 215/300 Read block 227 len 36864
25700 216/300 Write block 4127 len 24576 data: 7 5 2 2 2 2
25701 217/300 Flush
25702 218/300 Flush
25703 219/300 Read block 2304 len 24576
25704 220/300 Read block 837 len 40960
25705 221/300 Write block 2104 len 4096 data: 6
25706 222/300 Read block 2723 len 16384
25707 223/300 Write block 3556 len 36864 data: 8 5 6 6 5 6 5 5 4
25708 224/300 Read block 223 len 8192
25709 225/300 Read block 4004 len 40960
25710 226/300 Write block 3073 len 20480 data: 4 4 4 6 4
25711 227/300 Write block 568 len 28672 data: 6 6 4 3 3 3 4
25712 228/300 Read block 2371 len 28672
25713 229/300 Read block 581 len 40960
25714 230/300 Write block 1950 len 36864 data: 8 6 5 5 5 5 5 4 5
25715 231/300 Write block 283 len 40960 data: 3 3 4 5 6 6 7 5 4 6
25716 232/300 Read block 1688 len 12288
25717 233/300 Read block 1412 len 28672
25718 234/300 Flush
25719 235/300 Read block 1061 len 28672
25720 236/300 Read block 4161 len 8192
25721 237/300 Read block 2422 len 4096
25722 238/300 Read block 4266 len 28672
25723 239/300 Write block 722 len 36864 data: 6 6 6 5 4 5 7 6 5
25724 240/300 Write block 3058 len 8192 data: 5 4
25725 241/300 Flush
25726 242/300 Write block 1159 len 16384 data: 7 6 5 4
25727 243/300 Read block 1521 len 12288
25728 244/300 Read block 4253 len 32768
25729 245/300 Read block 2584 len 32768
25730 246/300 Write block 452 len 36864 data: 9 7 8 6 8 8 5 4 4
25731 247/300 Read block 3366 len 24576
25732 248/300 Flush
25733 249/300 Read block 4206 len 4096
25734 250/300 Write block 230 len 36864 data: 5 4 4 5 6 5 5 6 6
25735 251/300 Read block 4501 len 28672
25736 252/300 Write block 2393 len 4096 data: 2
25737 253/300 Read block 900 len 28672
25738 254/300 Read block 3157 len 20480
25739 255/300 Flush
25740 256/300 Flush
25741 257/300 Read block 2570 len 32768
25742 258/300 Read block 1207 len 4096
25743 259/300 Flush
25744 260/300 Flush
25745 261/300 Read block 4455 len 36864
25746 262/300 Read block 459 len 28672
25747 263/300 Read block 4585 len 4096
25748 264/300 Read block 276 len 8192
25749 265/300 Read block 1229 len 32768
25750 266/300 Read block 31 len 24576
25751 267/300 Write block 1454 len 32768 data: 2 3 3 3 3 3 3 3
25752 268/300 Write block 1890 len 16384 data: 5 5 4 2
25753 269/300 Read block 3862 len 36864
25754 270/300 Write block 4382 len 4096 data: 5
25755 271/300 Write block 2436 len 12288 data: 5 7 8
25756 272/300 Read block 895 len 8192
25757 273/300 Write block 2920 len 4096 data: 4
25758 274/300 Read block 4172 len 24576
25759 275/300 Write block 3587 len 8192 data: 5 4
25760 276/300 Flush
25761 277/300 Write block 2921 len 16384 data: 4 3 2 3
25762 278/300 Write block 2489 len 8192 data: 4 4
25763 279/300 Read block 4270 len 4096
25764 280/300 Read block 4135 len 16384
25765 281/300 Read block 3826 len 28672
25766 282/300 Read block 926 len 16384
25767 283/300 Read block 4555 len 28672
25768 284/300 Flush
25769 285/300 Write block 374 len 12288 data: 3 3 4
25770 286/300 Flush
25771 287/300 Write block 1366 len 16384 data: 5 4 4 4
25772 288/300 Flush
25773 289/300 Read block 2162 len 16384
25774 290/300 Read block 3879 len 4096
25775 291/300 Read block 4390 len 12288
25776 292/300 Read block 4373 len 36864
25777 293/300 Read block 1485 len 24576
25778 294/300 Write block 3674 len 20480 data: 6 6 5 6 5
25779 295/300 Read block 599 len 20480
25780 296/300 Write block 303 len 32768 data: 5 6 5 4 5 5 4 3
25781 297/300 Write block 3039 len 8192 data: 4 2
25782 298/300 Read block 4086 len 36864
25783 299/300 Read block 4947 len 12288
25784 300/300 Read block 1435 len 16384
25785 301/300 Read block 1666 len 20480
25786 Main task has work to do, go find it
25787 got dsc Start(2)
25788 start 2
25789 Nov 07 14:08:44.612 INFO request completed, latency_us: 239, response_code: 204, uri: /start/cid/2, method: POST, req_id: b574fe30-d212-4878-b5fe-b557223272ca, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
25790 Replay: started 2, returned:Ok(())
25791 ----------------------------------------------------------------
25792 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25793 States: Active Active Offline
25794 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25795 7652 Acked Read 9 Done Done Sent false
25796 7653 Acked Read 4 Done Done Sent false
25797 7654 Acked Read 10 Done Done Sent false
25798 7655 Acked Flush 0 Done Done Sent false
25799 7656 Acked Read 2 Done Done Sent false
25800 7657 Acked Write 2 Done Done Sent false
25801 7658 Acked Write 9 Done Done Sent false
25802 7659 Acked Read 2 Done Done Sent false
25803 7660 Acked Read 8 Done Done Sent false
25804 7661 Acked Write 8 Done Done Sent false
25805 7662 Acked Read 2 Done Done Sent false
25806 7663 Acked Write 2 Done Done Sent false
25807 7664 Acked Write 4 Done Done Sent false
25808 7665 Acked Write 9 Done Done Sent false
25809 7666 Acked Write 7 Done Done Sent false
25810 7667 Acked Flush 0 Done Done Sent false
25811 7668 Acked Flush 0 Done Done Sent false
25812 7669 Acked Write 9 Done Done Sent false
25813 7670 Acked Read 5 Done Done Sent false
25814 7671 Acked Write 4 Done Done Sent false
25815 7672 Acked Read 1 Done Done Sent false
25816 7673 Acked Read 8 Done Done Sent false
25817 7674 Acked Read 5 Done Done Sent false
25818 7675 Acked Read 3 Done Done Sent false
25819 7676 Acked Write 10 Done Done Sent false
25820 7677 Acked Write 5 Done Done Sent false
25821 7678 Acked Write 10 Done Done Sent false
25822 7679 Acked Read 7 Done Done Sent false
25823 7680 Acked Read 9 Done Done Sent false
25824 7681 Acked Write 2 Done Done Sent false
25825 7682 Acked Read 7 Done Done Sent false
25826 7683 Acked Write 7 Done Done Sent false
25827 7684 Acked Read 5 Done Done Sent false
25828 7685 Acked Write 2 Done Done Sent false
25829 7686 Acked Read 7 Done Done Sent false
25830 7687 Acked Write 9 Done Done Sent false
25831 7688 Acked Write 3 Done Done Sent false
25832 7689 Acked Read 6 Done Done Sent false
25833 7690 Acked Write 4 Done Done Sent false
25834 7691 Acked Write 3 Done Done Sent false
25835 7692 Acked Read 1 Done Done Sent false
25836 7693 Acked Read 7 Done Done Sent false
25837 7694 Acked Write 9 Done Done Sent false
25838 7695 Acked Flush 0 Done Done Sent false
25839 7696 Acked Read 3 Done Done Sent false
25840 7697 Acked Read 6 Done Done Sent false
25841 7698 Acked Flush 0 Done Done Sent false
25842 7699 Acked Write 10 Done Done Sent false
25843 7700 Acked Write 3 Done Done Sent false
25844 7701 Acked Flush 0 Done Done Sent false
25845 7702 Acked Read 3 Done Done Sent false
25846 7703 Acked Read 6 Done Done Sent false
25847 7704 Acked Write 2 Done Done Sent false
25848 7705 Acked Write 10 Done Done Sent false
25849 7706 Acked Write 9 Done Done Sent false
25850 7707 Acked Write 8 Done Done Sent false
25851 7708 Acked Read 8 Done Done Sent false
25852 7709 Acked Write 3 Done Done Sent false
25853 7710 Acked Read 1 Done Done Sent false
25854 7711 Acked Read 2 Done Done Sent false
25855 7712 Acked Write 8 Done Done Sent false
25856 7713 Acked Flush 0 Done Done Sent false
25857 7714 Acked Read 6 Done Done Sent false
25858 7715 Acked Write 8 Done Done Sent false
25859 7716 Acked Write 5 Done Done Sent false
25860 7717 Acked Write 7 Done Done Sent false
25861 7718 Acked Write 2 Done Done Sent false
25862 7719 Acked Read 7 Done Done Sent false
25863 7720 Acked Write 7 Done Done Sent false
25864 7721 Acked Read 2 Done Done Sent false
25865 7722 Acked Read 6 Done Done Sent false
25866 7723 Acked Write 6 Done Done Sent false
25867 7724 Acked Read 10 Done Done Sent false
25868 7725 Acked Read 10 Done Done Sent false
25869 7726 Acked Read 7 Done Done Sent false
25870 7727 Acked Read 3 Done Done Sent false
25871 7728 Acked Flush 0 Done Done Sent false
25872 7729 Acked Read 10 Done Done Sent false
25873 7730 Acked Flush 0 Done Done Sent false
25874 7731 Acked Read 6 Done Done Sent false
25875 7732 Acked Read 1 Done Done Sent false
25876 7733 Acked Read 9 Done Done Sent false
25877 7734 Acked Read 8 Done Done Sent false
25878 7735 Acked Read 1 Done Done Sent false
25879 7736 Acked Write 4 Done Done Sent false
25880 7737 Acked Write 1 Done Done Sent false
25881 7738 Acked Flush 0 Done Done Sent false
25882 7739 Acked Write 3 Done Done Sent false
25883 7740 Acked Write 5 Done Done Sent false
25884 7741 Acked Read 10 Done Done Sent false
25885 7742 Acked Read 10 Done Done Sent false
25886 7743 Acked Write 6 Done Done Sent false
25887 7744 Acked Flush 0 Done Done Sent false
25888 7745 Acked Flush 0 Done Done Sent false
25889 7746 Acked Read 8 Done Done Sent false
25890 7747 Acked Write 9 Done Done Sent false
25891 7748 Acked Read 7 Done Done Sent false
25892 7749 Acked Write 2 Done Done Sent false
25893 7750 Acked Read 6 Done Done Sent false
25894 7751 Acked Read 4 Done Done Sent false
25895 7752 Acked Read 7 Done Done Sent false
25896 7753 Acked Flush 0 Done Done Sent false
25897 7754 Acked Write 9 Done Done Sent false
25898 7755 Acked Write 8 Done Done Sent false
25899 7756 Acked Read 7 Done Done Sent false
25900 7757 Acked Flush 0 Done Done Sent false
25901 7758 Acked Write 3 Done Done Sent false
25902 7759 Acked Flush 0 Done Done Sent false
25903 7760 Acked Write 2 Done Done Sent false
25904 7761 Acked Write 9 Done Done Sent false
25905 7762 Acked Read 7 Done Done Sent false
25906 7763 Acked Write 8 Done Done Sent false
25907 7764 Acked Flush 0 Done Done Sent false
25908 7765 Acked Read 3 Done Done Sent false
25909 7766 Acked Read 8 Done Done Sent false
25910 7767 Acked Read 7 Done Done Sent false
25911 7768 Acked Write 10 Done Done Sent false
25912 7769 Acked Read 7 Done Done Sent false
25913 7770 Acked Write 7 Done Done Sent false
25914 7771 Acked Write 1 Done Done Sent false
25915 7772 Acked Flush 0 Done Done Sent false
25916 7773 Acked Write 10 Done Done Sent false
25917 7774 Acked Write 3 Done Done Sent false
25918 7775 Acked Write 5 Done Done Sent false
25919 7776 Acked Write 6 Done Done Sent false
25920 7777 Acked Write 9 Done Done Sent false
25921 7778 Acked Read 7 Done Done Sent false
25922 7779 Acked Read 8 Done Done Sent false
25923 7780 Acked Write 8 Done Done Sent false
25924 7781 Acked Write 10 Done Done Sent false
25925 7782 Acked Read 6 Done Done Sent false
25926 7783 Acked Flush 0 Done Done Sent false
25927 7784 Acked Read 2 Done Done Sent false
25928 7785 Acked Write 10 Done Done Sent false
25929 7786 Acked Read 1 Done Done Sent false
25930 7787 Acked Read 10 Done Done Sent false
25931 7788 Acked Flush 0 Done Done Sent false
25932 7789 Acked Read 2 Done Done Sent false
25933 7790 Acked Read 4 Done Done Sent false
25934 7791 Acked Read 1 Done Done Sent false
25935 7792 Acked Write 4 Done Done Sent false
25936 7793 Acked Write 4 Done Done Sent false
25937 7794 Acked Write 4 Done Done Sent false
25938 7795 Acked Read 8 Done Done Sent false
25939 7796 Acked Write 2 Done Done Sent false
25940 7797 Acked Write 1 Done Done Sent false
25941 7798 Acked Write 6 Done Done Sent false
25942 7799 Acked Write 4 Done Done Sent false
25943 7800 Acked Read 2 Done Done Sent false
25944 7801 Acked Read 5 Done Done Sent false
25945 7802 Acked Read 5 Done Done Sent false
25946 7803 Acked Flush 0 Done Done Sent false
25947 7804 Acked Read 2 Done Done Sent false
25948 7805 Acked Write 5 Done Done Sent false
25949 7806 Acked Write 9 Done Done Sent false
25950 7807 Acked Write 7 Done Done Sent false
25951 7808 Acked Flush 0 Done Done Sent false
25952 7809 Acked Read 7 Done Done Sent false
25953 7810 Acked Read 9 Done Done Sent false
25954 7811 Acked Write 9 Done Done Sent false
25955 7812 Acked Flush 0 Done Done Sent false
25956 7813 Acked Read 4 Done Done Sent false
25957 7814 Acked Read 5 Done Done Sent false
25958 7815 Acked Read 8 Done Done Sent false
25959 7816 Acked Read 9 Done Done Sent false
25960 7817 Acked Read 4 Done Done Sent false
25961 7818 Acked Read 3 Done Done Sent false
25962 7819 Acked Write 10 Done Done Sent false
25963 7820 Acked Read 6 Done Done Sent false
25964 7821 Acked Read 4 Done Done Sent false
25965 7822 Acked Flush 0 Done Done Sent false
25966 7823 Acked Read 5 Done Done Sent false
25967 7824 Acked Read 3 Done Done Sent false
25968 7825 Acked Read 10 Done Done Sent false
25969 7826 Acked Flush 0 Done Done Sent false
25970 7827 Acked Read 2 Done Done Sent false
25971 7828 Acked Read 8 Done Done Sent false
25972 7829 Acked Read 6 Done Done Sent false
25973 7830 Acked Write 9 Done Done Sent false
25974 7831 Acked Write 4 Done Done Sent false
25975 7832 Acked Read 2 Done Done Sent false
25976 7833 Acked Write 3 Done Done Sent false
25977 7834 Acked Flush 0 Done Done Sent false
25978 7835 Acked Read 8 Done Done Sent false
25979 7836 Acked Read 3 Done Done Sent false
25980 7837 Acked Write 9 Done Done Sent false
25981 7838 Acked Write 7 Done Done Sent false
25982 7839 Acked Read 6 Done Done Sent false
25983 7840 Acked Read 8 Done Done Sent false
25984 7841 Acked Read 2 Done Done Sent false
25985 7842 Acked Write 6 Done Done Sent false
25986 7843 Acked Read 5 Done Done Sent false
25987 7844 Acked Write 5 Done Done Sent false
25988 7845 Acked Write 10 Done Done Sent false
25989 7846 Acked Write 8 Done Done Sent false
25990 7847 Acked Read 7 Done Done Sent false
25991 7848 Acked Read 1 Done Done Sent false
25992 7849 Acked Read 4 Done Done Sent false
25993 7850 Acked Write 9 Done Done Sent false
25994 7851 Acked Read 3 Done Done Sent false
25995 7852 Acked Write 8 Done Done Sent false
25996 7853 Acked Read 1 Done Done Sent false
25997 7854 Acked Read 8 Done Done Sent false
25998 7855 Acked Write 3 Done Done Sent false
25999 7856 Acked Read 1 Done Done Sent false
26000 7857 Acked Read 9 Done Done Sent false
26001 7858 Acked Write 4 Done Done Sent false
26002 7859 Acked Read 7 Done Done Sent false
26003 7860 Acked Read 3 Done Done Sent false
26004 7861 Acked Read 4 Done Done Sent false
26005 7862 Acked Read 3 Done Done Sent false
26006 7863 Acked Write 1 Done Done Sent false
26007 7864 Acked Write 10 Done Done Sent false
26008 7865 Acked Write 9 Done Done Sent false
26009 7866 Acked Read 9 Done Done Sent false
26010 7867 Acked Read 9 Done Done Sent false
26011 7868 Acked Write 6 Done Done Sent false
26012 7869 Acked Flush 0 Done Done Sent false
26013 7870 Acked Flush 0 Done Done Sent false
26014 7871 Acked Read 6 Done Done Sent false
26015 7872 Acked Read 10 Done Done Sent false
26016 7873 Acked Write 1 Done Done Sent false
26017 7874 Acked Read 4 Done Done Sent false
26018 7875 Acked Write 9 Done Done Sent false
26019 7876 Acked Read 2 Done Done Sent false
26020 7877 Acked Read 10 Done Done Sent false
26021 7878 Acked Write 5 Done Done Sent false
26022 7879 Acked Write 7 Done Done Sent false
26023 7880 Acked Read 7 Done Done Sent false
26024 7881 Acked Read 10 Done Done Sent false
26025 7882 Acked Write 9 Done Done Sent false
26026 7883 Acked Write 10 Done Done Sent false
26027 7884 Acked Read 3 Done Done Sent false
26028 7885 Acked Read 7 Done Done Sent false
26029 7886 Acked Flush 0 Done Done Sent false
26030 7887 Acked Read 7 Done Done Sent false
26031 7888 Acked Read 2 Done Done Sent false
26032 7889 Acked Read 1 Done Done Sent false
26033 7890 Acked Read 7 Done Done Sent false
26034 7891 Acked Write 9 Done Done Sent false
26035 7892 Acked Write 2 Done Done Sent false
26036 7893 Acked Flush 0 Done Done Sent false
26037 7894 Acked Write 4 Done Done Sent false
26038 7895 Acked Read 3 Done Done Sent false
26039 7896 Acked Read 8 Done Done Sent false
26040 7897 Acked Read 8 Done Done Sent false
26041 7898 Acked Write 9 Done Done Sent false
26042 7899 Acked Read 6 Done Done Sent false
26043 7900 Acked Flush 0 Done Done Sent false
26044 7901 Acked Read 1 Done Done Sent false
26045 7902 Acked Write 9 Done Done Sent false
26046 7903 Acked Read 7 Done Done Sent false
26047 7904 Acked Write 1 Done Done Sent false
26048 7905 Acked Read 7 Done Done Sent false
26049 7906 Acked Read 5 Done Done Sent false
26050 7907 Acked Flush 0 Done Done Sent false
26051 7908 Acked Flush 0 Done Done Sent false
26052 7909 Acked Read 8 Done Done Sent false
26053 7910 Acked Read 1 Done Done Sent false
26054 7911 Acked Flush 0 Done Done Sent false
26055 7912 Acked Flush 0 Done Done Sent false
26056 7913 Acked Read 9 Done Done Sent false
26057 7914 Acked Read 7 Done Done Sent false
26058 7915 Acked Read 1 Done Done Sent false
26059 7916 Acked Read 2 Done Done Sent false
26060 7917 Acked Read 8 Done Done Sent false
26061 7918 Acked Read 6 Done Done Sent false
26062 7919 Acked Write 8 Done Done Sent false
26063 7920 Acked Write 4 Done Done Sent false
26064 7921 Acked Read 9 Done Done Sent false
26065 7922 Acked Write 1 Done Done Sent false
26066 7923 Acked Write 3 Done Done Sent false
26067 7924 Acked Read 2 Done Done Sent false
26068 7925 Acked Write 1 Done Done Sent false
26069 7926 Acked Read 6 Done Done Sent false
26070 7927 Acked Write 2 Done Done Sent false
26071 7928 Acked Flush 0 Done Done Sent false
26072 7929 Acked Write 4 Done Done Sent false
26073 7930 Acked Write 2 Done Done Sent false
26074 7931 Acked Read 1 Done Done Sent false
26075 7932 Acked Read 4 Done Done Sent false
26076 7933 Acked Read 7 Done Done Sent false
26077 7934 Acked Read 4 Done Done Sent false
26078 7935 Acked Read 7 Done Done Sent false
26079 7936 Acked Flush 0 Done Done Sent false
26080 7937 Acked Write 3 Done Done Sent false
26081 7938 Acked Flush 0 Done Done Sent false
26082 7939 Acked Write 4 Done Done Sent false
26083 7940 Acked Flush 0 Done Done Sent false
26084 7941 Acked Read 4 Done Done Sent false
26085 7942 Acked Read 1 Done Done Sent false
26086 7943 Acked Read 3 Done Done Sent false
26087 7944 Acked Read 9 Done Done Sent false
26088 7945 Acked Read 6 Done Done Sent false
26089 7946 Acked Write 5 Done Done Sent false
26090 7947 Acked Read 5 Done Done Sent false
26091 7948 Acked Write 8 Done Done Sent false
26092 7949 Acked Write 2 Done Done Sent false
26093 7950 Acked Read 9 Done Done Sent false
26094 7951 Acked Read 3 Done Done Sent false
26095 7952 Acked Read 4 Done Done Sent false
26096 7953 Acked Read 5 Done Done Sent false
26097 STATES DS:0 DS:1 DS:2 TOTAL
26098 Sent 0 0 302 302
26099 Done 302 302 0 604
26100 Skipped 0 0 0 0
26101 Error 0 0 0 0
26102 Last Flush: 7940 7940 7651
26103 Downstairs last five completed: 7651 7650 7649 7648 7647
26104 Upstairs last five completed: 7953 7952 7951 7950 7949
26105 CLIENT: Up:0 ds:302 act:2
26106 [8830] got start action
26107 [8830] I am going to restart
26108 Starting downstairs at port 8830
26109 Make output file at "/tmp/dsc/downstairs-8830.txt"
26110 [8830][2] reports Starting
26111 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1222)
26112 [8830][2] reports Running
26113 ----------------------------------------------------------------
26114 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
26115 States: Active Active Offline
26116 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26117 7652 Acked Read 9 Done Done Sent false
26118 7653 Acked Read 4 Done Done Sent false
26119 7654 Acked Read 10 Done Done Sent false
26120 7655 Acked Flush 0 Done Done Sent false
26121 7656 Acked Read 2 Done Done Sent false
26122 7657 Acked Write 2 Done Done Sent false
26123 7658 Acked Write 9 Done Done Sent false
26124 7659 Acked Read 2 Done Done Sent false
26125 7660 Acked Read 8 Done Done Sent false
26126 7661 Acked Write 8 Done Done Sent false
26127 7662 Acked Read 2 Done Done Sent false
26128 7663 Acked Write 2 Done Done Sent false
26129 7664 Acked Write 4 Done Done Sent false
26130 7665 Acked Write 9 Done Done Sent false
26131 7666 Acked Write 7 Done Done Sent false
26132 7667 Acked Flush 0 Done Done Sent false
26133 7668 Acked Flush 0 Done Done Sent false
26134 7669 Acked Write 9 Done Done Sent false
26135 7670 Acked Read 5 Done Done Sent false
26136 7671 Acked Write 4 Done Done Sent false
26137 7672 Acked Read 1 Done Done Sent false
26138 7673 Acked Read 8 Done Done Sent false
26139 7674 Acked Read 5 Done Done Sent false
26140 7675 Acked Read 3 Done Done Sent false
26141 7676 Acked Write 10 Done Done Sent false
26142 7677 Acked Write 5 Done Done Sent false
26143 7678 Acked Write 10 Done Done Sent false
26144 7679 Acked Read 7 Done Done Sent false
26145 7680 Acked Read 9 Done Done Sent false
26146 7681 Acked Write 2 Done Done Sent false
26147 7682 Acked Read 7 Done Done Sent false
26148 7683 Acked Write 7 Done Done Sent false
26149 7684 Acked Read 5 Done Done Sent false
26150 7685 Acked Write 2 Done Done Sent false
26151 7686 Acked Read 7 Done Done Sent false
26152 7687 Acked Write 9 Done Done Sent false
26153 7688 Acked Write 3 Done Done Sent false
26154 7689 Acked Read 6 Done Done Sent false
26155 7690 Acked Write 4 Done Done Sent false
26156 7691 Acked Write 3 Done Done Sent false
26157 7692 Acked Read 1 Done Done Sent false
26158 7693 Acked Read 7 Done Done Sent false
26159 7694 Acked Write 9 Done Done Sent false
26160 7695 Acked Flush 0 Done Done Sent false
26161 7696 Acked Read 3 Done Done Sent false
26162 7697 Acked Read 6 Done Done Sent false
26163 7698 Acked Flush 0 Done Done Sent false
26164 7699 Acked Write 10 Done Done Sent false
26165 7700 Acked Write 3 Done Done Sent false
26166 7701 Acked Flush 0 Done Done Sent false
26167 7702 Acked Read 3 Done Done Sent false
26168 7703 Acked Read 6 Done Done Sent false
26169 7704 Acked Write 2 Done Done Sent false
26170 7705 Acked Write 10 Done Done Sent false
26171 7706 Acked Write 9 Done Done Sent false
26172 7707 Acked Write 8 Done Done Sent false
26173 7708 Acked Read 8 Done Done Sent false
26174 7709 Acked Write 3 Done Done Sent false
26175 7710 Acked Read 1 Done Done Sent false
26176 7711 Acked Read 2 Done Done Sent false
26177 7712 Acked Write 8 Done Done Sent false
26178 7713 Acked Flush 0 Done Done Sent false
26179 7714 Acked Read 6 Done Done Sent false
26180 7715 Acked Write 8 Done Done Sent false
26181 7716 Acked Write 5 Done Done Sent false
26182 7717 Acked Write 7 Done Done Sent false
26183 7718 Acked Write 2 Done Done Sent false
26184 7719 Acked Read 7 Done Done Sent false
26185 7720 Acked Write 7 Done Done Sent false
26186 7721 Acked Read 2 Done Done Sent false
26187 7722 Acked Read 6 Done Done Sent false
26188 7723 Acked Write 6 Done Done Sent false
26189 7724 Acked Read 10 Done Done Sent false
26190 7725 Acked Read 10 Done Done Sent false
26191 7726 Acked Read 7 Done Done Sent false
26192 7727 Acked Read 3 Done Done Sent false
26193 7728 Acked Flush 0 Done Done Sent false
26194 7729 Acked Read 10 Done Done Sent false
26195 7730 Acked Flush 0 Done Done Sent false
26196 7731 Acked Read 6 Done Done Sent false
26197 7732 Acked Read 1 Done Done Sent false
26198 7733 Acked Read 9 Done Done Sent false
26199 7734 Acked Read 8 Done Done Sent false
26200 7735 Acked Read 1 Done Done Sent false
26201 7736 Acked Write 4 Done Done Sent false
26202 7737 Acked Write 1 Done Done Sent false
26203 7738 Acked Flush 0 Done Done Sent false
26204 7739 Acked Write 3 Done Done Sent false
26205 7740 Acked Write 5 Done Done Sent false
26206 7741 Acked Read 10 Done Done Sent false
26207 7742 Acked Read 10 Done Done Sent false
26208 7743 Acked Write 6 Done Done Sent false
26209 7744 Acked Flush 0 Done Done Sent false
26210 7745 Acked Flush 0 Done Done Sent false
26211 7746 Acked Read 8 Done Done Sent false
26212 7747 Acked Write 9 Done Done Sent false
26213 7748 Acked Read 7 Done Done Sent false
26214 7749 Acked Write 2 Done Done Sent false
26215 7750 Acked Read 6 Done Done Sent false
26216 7751 Acked Read 4 Done Done Sent false
26217 7752 Acked Read 7 Done Done Sent false
26218 7753 Acked Flush 0 Done Done Sent false
26219 7754 Acked Write 9 Done Done Sent false
26220 7755 Acked Write 8 Done Done Sent false
26221 7756 Acked Read 7 Done Done Sent false
26222 7757 Acked Flush 0 Done Done Sent false
26223 7758 Acked Write 3 Done Done Sent false
26224 7759 Acked Flush 0 Done Done Sent false
26225 7760 Acked Write 2 Done Done Sent false
26226 7761 Acked Write 9 Done Done Sent false
26227 7762 Acked Read 7 Done Done Sent false
26228 7763 Acked Write 8 Done Done Sent false
26229 7764 Acked Flush 0 Done Done Sent false
26230 7765 Acked Read 3 Done Done Sent false
26231 7766 Acked Read 8 Done Done Sent false
26232 7767 Acked Read 7 Done Done Sent false
26233 7768 Acked Write 10 Done Done Sent false
26234 7769 Acked Read 7 Done Done Sent false
26235 7770 Acked Write 7 Done Done Sent false
26236 7771 Acked Write 1 Done Done Sent false
26237 7772 Acked Flush 0 Done Done Sent false
26238 7773 Acked Write 10 Done Done Sent false
26239 7774 Acked Write 3 Done Done Sent false
26240 7775 Acked Write 5 Done Done Sent false
26241 7776 Acked Write 6 Done Done Sent false
26242 7777 Acked Write 9 Done Done Sent false
26243 7778 Acked Read 7 Done Done Sent false
26244 7779 Acked Read 8 Done Done Sent false
26245 7780 Acked Write 8 Done Done Sent false
26246 7781 Acked Write 10 Done Done Sent false
26247 7782 Acked Read 6 Done Done Sent false
26248 7783 Acked Flush 0 Done Done Sent false
26249 7784 Acked Read 2 Done Done Sent false
26250 7785 Acked Write 10 Done Done Sent false
26251 7786 Acked Read 1 Done Done Sent false
26252 7787 Acked Read 10 Done Done Sent false
26253 7788 Acked Flush 0 Done Done Sent false
26254 7789 Acked Read 2 Done Done Sent false
26255 7790 Acked Read 4 Done Done Sent false
26256 7791 Acked Read 1 Done Done Sent false
26257 7792 Acked Write 4 Done Done Sent false
26258 7793 Acked Write 4 Done Done Sent false
26259 7794 Acked Write 4 Done Done Sent false
26260 7795 Acked Read 8 Done Done Sent false
26261 7796 Acked Write 2 Done Done Sent false
26262 7797 Acked Write 1 Done Done Sent false
26263 7798 Acked Write 6 Done Done Sent false
26264 7799 Acked Write 4 Done Done Sent false
26265 7800 Acked Read 2 Done Done Sent false
26266 7801 Acked Read 5 Done Done Sent false
26267 7802 Acked Read 5 Done Done Sent false
26268 7803 Acked Flush 0 Done Done Sent false
26269 7804 Acked Read 2 Done Done Sent false
26270 7805 Acked Write 5 Done Done Sent false
26271 7806 Acked Write 9 Done Done Sent false
26272 7807 Acked Write 7 Done Done Sent false
26273 7808 Acked Flush 0 Done Done Sent false
26274 7809 Acked Read 7 Done Done Sent false
26275 7810 Acked Read 9 Done Done Sent false
26276 7811 Acked Write 9 Done Done Sent false
26277 7812 Acked Flush 0 Done Done Sent false
26278 7813 Acked Read 4 Done Done Sent false
26279 7814 Acked Read 5 Done Done Sent false
26280 7815 Acked Read 8 Done Done Sent false
26281 7816 Acked Read 9 Done Done Sent false
26282 7817 Acked Read 4 Done Done Sent false
26283 7818 Acked Read 3 Done Done Sent false
26284 7819 Acked Write 10 Done Done Sent false
26285 7820 Acked Read 6 Done Done Sent false
26286 7821 Acked Read 4 Done Done Sent false
26287 7822 Acked Flush 0 Done Done Sent false
26288 7823 Acked Read 5 Done Done Sent false
26289 7824 Acked Read 3 Done Done Sent false
26290 7825 Acked Read 10 Done Done Sent false
26291 7826 Acked Flush 0 Done Done Sent false
26292 7827 Acked Read 2 Done Done Sent false
26293 7828 Acked Read 8 Done Done Sent false
26294 7829 Acked Read 6 Done Done Sent false
26295 7830 Acked Write 9 Done Done Sent false
26296 7831 Acked Write 4 Done Done Sent false
26297 7832 Acked Read 2 Done Done Sent false
26298 7833 Acked Write 3 Done Done Sent false
26299 7834 Acked Flush 0 Done Done Sent false
26300 7835 Acked Read 8 Done Done Sent false
26301 7836 Acked Read 3 Done Done Sent false
26302 7837 Acked Write 9 Done Done Sent false
26303 7838 Acked Write 7 Done Done Sent false
26304 7839 Acked Read 6 Done Done Sent false
26305 7840 Acked Read 8 Done Done Sent false
26306 7841 Acked Read 2 Done Done Sent false
26307 7842 Acked Write 6 Done Done Sent false
26308 7843 Acked Read 5 Done Done Sent false
26309 7844 Acked Write 5 Done Done Sent false
26310 7845 Acked Write 10 Done Done Sent false
26311 7846 Acked Write 8 Done Done Sent false
26312 7847 Acked Read 7 Done Done Sent false
26313 7848 Acked Read 1 Done Done Sent false
26314 7849 Acked Read 4 Done Done Sent false
26315 7850 Acked Write 9 Done Done Sent false
26316 7851 Acked Read 3 Done Done Sent false
26317 7852 Acked Write 8 Done Done Sent false
26318 7853 Acked Read 1 Done Done Sent false
26319 7854 Acked Read 8 Done Done Sent false
26320 7855 Acked Write 3 Done Done Sent false
26321 7856 Acked Read 1 Done Done Sent false
26322 7857 Acked Read 9 Done Done Sent false
26323 7858 Acked Write 4 Done Done Sent false
26324 7859 Acked Read 7 Done Done Sent false
26325 7860 Acked Read 3 Done Done Sent false
26326 7861 Acked Read 4 Done Done Sent false
26327 7862 Acked Read 3 Done Done Sent false
26328 7863 Acked Write 1 Done Done Sent false
26329 7864 Acked Write 10 Done Done Sent false
26330 7865 Acked Write 9 Done Done Sent false
26331 7866 Acked Read 9 Done Done Sent false
26332 7867 Acked Read 9 Done Done Sent false
26333 7868 Acked Write 6 Done Done Sent false
26334 7869 Acked Flush 0 Done Done Sent false
26335 7870 Acked Flush 0 Done Done Sent false
26336 7871 Acked Read 6 Done Done Sent false
26337 7872 Acked Read 10 Done Done Sent false
26338 7873 Acked Write 1 Done Done Sent false
26339 7874 Acked Read 4 Done Done Sent false
26340 7875 Acked Write 9 Done Done Sent false
26341 7876 Acked Read 2 Done Done Sent false
26342 7877 Acked Read 10 Done Done Sent false
26343 7878 Acked Write 5 Done Done Sent false
26344 7879 Acked Write 7 Done Done Sent false
26345 7880 Acked Read 7 Done Done Sent false
26346 7881 Acked Read 10 Done Done Sent false
26347 7882 Acked Write 9 Done Done Sent false
26348 7883 Acked Write 10 Done Done Sent false
26349 7884 Acked Read 3 Done Done Sent false
26350 7885 Acked Read 7 Done Done Sent false
26351 7886 Acked Flush 0 Done Done Sent false
26352 7887 Acked Read 7 Done Done Sent false
26353 7888 Acked Read 2 Done Done Sent false
26354 7889 Acked Read 1 Done Done Sent false
26355 7890 Acked Read 7 Done Done Sent false
26356 7891 Acked Write 9 Done Done Sent false
26357 7892 Acked Write 2 Done Done Sent false
26358 7893 Acked Flush 0 Done Done Sent false
26359 7894 Acked Write 4 Done Done Sent false
26360 7895 Acked Read 3 Done Done Sent false
26361 7896 Acked Read 8 Done Done Sent false
26362 7897 Acked Read 8 Done Done Sent false
26363 7898 Acked Write 9 Done Done Sent false
26364 7899 Acked Read 6 Done Done Sent false
26365 7900 Acked Flush 0 Done Done Sent false
26366 7901 Acked Read 1 Done Done Sent false
26367 7902 Acked Write 9 Done Done Sent false
26368 7903 Acked Read 7 Done Done Sent false
26369 7904 Acked Write 1 Done Done Sent false
26370 7905 Acked Read 7 Done Done Sent false
26371 7906 Acked Read 5 Done Done Sent false
26372 7907 Acked Flush 0 Done Done Sent false
26373 7908 Acked Flush 0 Done Done Sent false
26374 7909 Acked Read 8 Done Done Sent false
26375 7910 Acked Read 1 Done Done Sent false
26376 7911 Acked Flush 0 Done Done Sent false
26377 7912 Acked Flush 0 Done Done Sent false
26378 7913 Acked Read 9 Done Done Sent false
26379 7914 Acked Read 7 Done Done Sent false
26380 7915 Acked Read 1 Done Done Sent false
26381 7916 Acked Read 2 Done Done Sent false
26382 7917 Acked Read 8 Done Done Sent false
26383 7918 Acked Read 6 Done Done Sent false
26384 7919 Acked Write 8 Done Done Sent false
26385 7920 Acked Write 4 Done Done Sent false
26386 7921 Acked Read 9 Done Done Sent false
26387 7922 Acked Write 1 Done Done Sent false
26388 7923 Acked Write 3 Done Done Sent false
26389 7924 Acked Read 2 Done Done Sent false
26390 7925 Acked Write 1 Done Done Sent false
26391 7926 Acked Read 6 Done Done Sent false
26392 7927 Acked Write 2 Done Done Sent false
26393 7928 Acked Flush 0 Done Done Sent false
26394 7929 Acked Write 4 Done Done Sent false
26395 7930 Acked Write 2 Done Done Sent false
26396 7931 Acked Read 1 Done Done Sent false
26397 7932 Acked Read 4 Done Done Sent false
26398 7933 Acked Read 7 Done Done Sent false
26399 7934 Acked Read 4 Done Done Sent false
26400 7935 Acked Read 7 Done Done Sent false
26401 7936 Acked Flush 0 Done Done Sent false
26402 7937 Acked Write 3 Done Done Sent false
26403 7938 Acked Flush 0 Done Done Sent false
26404 7939 Acked Write 4 Done Done Sent false
26405 7940 Acked Flush 0 Done Done Sent false
26406 7941 Acked Read 4 Done Done Sent false
26407 7942 Acked Read 1 Done Done Sent false
26408 7943 Acked Read 3 Done Done Sent false
26409 7944 Acked Read 9 Done Done Sent false
26410 7945 Acked Read 6 Done Done Sent false
26411 7946 Acked Write 5 Done Done Sent false
26412 7947 Acked Read 5 Done Done Sent false
26413 7948 Acked Write 8 Done Done Sent false
26414 7949 Acked Write 2 Done Done Sent false
26415 7950 Acked Read 9 Done Done Sent false
26416 7951 Acked Read 3 Done Done Sent false
26417 7952 Acked Read 4 Done Done Sent false
26418 7953 Acked Read 5 Done Done Sent false
26419 7954 Acked Flush 0 Done Done Sent false
26420 STATES DS:0 DS:1 DS:2 TOTAL
26421 Sent 0 0 303 303
26422 Done 303 303 0 606
26423 Skipped 0 0 0 0
26424 Error 0 0 0 0
26425 Last Flush: 7954 7954 7651
26426 Downstairs last five completed: 7651 7650 7649 7648 7647
26427 Upstairs last five completed: 7954 7953 7952 7951 7950
26428 CLIENT: Up:0 ds:303 act:2
264292024-11-07T14:08:50.319ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264302024-11-07T14:08:50.319ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41224) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264312024-11-07T14:08:50.320ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264322024-11-07T14:08:50.321ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264332024-11-07T14:08:50.321ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264342024-11-07T14:08:50.321ZINFOcrucible: send last flush ID to this DS: 7651 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264352024-11-07T14:08:50.321ZINFOcrucible: Replied this last flush ID: 7651 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264362024-11-07T14:08:50.321ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264372024-11-07T14:08:50.321ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264382024-11-07T14:08:50.321ZINFOcrucible: [2] client re-new 303 jobs since flush 7651 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264392024-11-07T14:08:50.321ZINFOcrucible: [2] Replayed 303 jobs since flush: 7651 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
26440 ----------------------------------------------------------------
26441 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26442 Downstairs last five completed: 7954 7953 7952 7951 7950
26443 Upstairs last five completed: 7954 7953 7952 7951 7950
26444 CLIENT: Up:0 ds:0 act:3
26445 Replay: All jobs finished
26446 Main task has work to do, go find it
26447 got dsc Stop(1)
26448 stop 1
26449 Nov 07 14:08:52.617[8820] Got stop action so:false kr:false
26450 INFO request completed, latency_us: 365, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 35781315-32b8-41fd-8c5c-3d88a5beb5f2, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
264512024-11-07T14:08:52.617ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264522024-11-07T14:08:52.617ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264532024-11-07T14:08:52.617ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264542024-11-07T14:08:52.617ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264552024-11-07T14:08:52.617ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
264562024-11-07T14:08:52.617ZINFOcrucible: client task is exiting = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
26457 Nov 07 14:08:52.618 INFO request completed, latency_us: 224, response_code: 200, uri: /state/cid/1, method: GET, req_id: 02a66f14-fa8e-4a35-82bd-647ccb54831c, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
26458 [8820] Exited with: ExitStatus(unix_wait_status(9))
26459 [8820][1] reports Exit
26460 Nov 07 14:08:56.620 INFO request completed, latency_us: 231, response_code: 200, uri: /state/cid/1, method: GET, req_id: 590e0446-803b-4143-abf7-43fdb31dc059, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
26461 001/300 Read block 4147 len 40960
26462 002/300 Write block 2984 len 12288 data: 5 5 5
26463 003/300 Write block 4723 len 28672 data: 5 6 6 4 7 8 9
26464 004/300 Write block 268 len 12288 data: 4 4 4
26465 005/300 Read block 2700 len 12288
26466 006/300 Write block 4606 len 4096 data: 4
26467 007/300 Read block 2977 len 4096
26468 008/300 Read block 4082 len 4096
26469 009/300 Write block 2809 len 20480 data: 6 4 3 3 3
26470 010/300 Read block 4397 len 36864
26471 011/300 Read block 3841 len 4096
26472 012/300 Read block 574 len 40960
26473 013/300 Flush
26474 014/300 Read block 4609 len 24576
26475 015/300 Read block 2904 len 20480
26476 016/300 Read block 3519 len 16384
26477 017/300 Read block 3504 len 28672
26478 018/300 Write block 2971 len 24576 data: 7 7 7 7 6 7
26479 019/300 Read block 1073 len 16384
26480 020/300 Write block 232 len 20480 data: 5 6 7 6 6
26481 021/300 Write block 2560 len 24576 data: 5 4 3 4 4 5
26482 022/300 Write block 981 len 4096 data: 7
26483 023/300 Read block 2801 len 8192
26484 024/300 Read block 2191 len 40960
26485 025/300 Read block 2872 len 36864
26486 026/300 Write block 2891 len 20480 data: 3 3 3 3 3
26487 027/300 Read block 3569 len 4096
26488 028/300 Read block 2049 len 28672
26489 029/300 Read block 4577 len 8192
26490 030/300 Read block 2670 len 20480
26491 031/300 Flush
26492 032/300 Read block 1781 len 24576
26493 033/300 Write block 899 len 4096 data: 11
26494 034/300 Read block 4427 len 16384
26495 035/300 Read block 2667 len 24576
26496 036/300 Read block 1358 len 36864
26497 037/300 Read block 137 len 28672
26498 038/300 Write block 2698 len 28672 data: 6 6 6 5 5 5 4
26499 039/300 Write block 3804 len 24576 data: 8 6 5 4 5 4
26500 040/300 Write block 2097 len 12288 data: 2 2 2
26501 041/300 Write block 2685 len 4096 data: 5
26502 042/300 Read block 3739 len 32768
26503 043/300 Read block 1045 len 4096
26504 044/300 Read block 1187 len 8192
26505 045/300 Read block 2178 len 16384
26506 046/300 Read block 3514 len 24576
26507 047/300 Read block 1347 len 4096
26508 048/300 Read block 2167 len 36864
26509 049/300 Flush
26510 050/300 Write block 4841 len 12288 data: 7 9 8
26511 051/300 Write block 4405 len 28672 data: 4 4 4 4 3 3 4
26512 052/300 Read block 1532 len 28672
26513 053/300 Flush
26514 054/300 Read block 4055 len 32768
26515 055/300 Flush
26516 056/300 Write block 4174 len 8192 data: 8 6
26517 057/300 Write block 4675 len 40960 data: 7 6 5 5 4 3 2 2 3 3
26518 058/300 Read block 4367 len 28672
26519 059/300 Write block 3251 len 8192 data: 2 2
26520 060/300 Write block 2284 len 28672 data: 4 4 4 3 3 3 3
26521 061/300 Read block 4643 len 20480
26522 062/300 Read block 958 len 12288
26523 063/300 Flush
26524 064/300 Read block 1354 len 4096
26525 065/300 Flush
26526 066/300 Read block 48 len 40960
26527 067/300 Read block 3680 len 24576
26528 068/300 Read block 3214 len 36864
26529 069/300 Write block 2715 len 16384 data: 3 3 3 2
26530 070/300 Read block 333 len 12288
26531 071/300 Write block 2917 len 24576 data: 5 5 5 5 5 4
26532 072/300 Read block 624 len 28672
26533 073/300 Read block 3541 len 40960
26534 074/300 Read block 1154 len 40960
26535 075/300 Write block 1949 len 40960 data: 7 9 7 6 6 6 6 6 5 6
26536 076/300 Read block 4276 len 24576
26537 077/300 Write block 4173 len 36864 data: 8 9 7 6 7 8 7 6 6
26538 078/300 Read block 4291 len 32768
26539 079/300 Read block 1652 len 16384
26540 080/300 Read block 3607 len 40960
26541 081/300 Read block 2162 len 12288
26542 082/300 Read block 1742 len 20480
26543 083/300 Flush
26544 084/300 Read block 4856 len 36864
26545 085/300 Read block 1415 len 32768
26546 086/300 Write block 1916 len 40960 data: 6 5 5 7 6 6 5 4 4 5
26547 087/300 Write block 4516 len 20480 data: 2 2 2 3 4
26548 088/300 Read block 556 len 28672
26549 089/300 Read block 4259 len 12288
26550 090/300 Read block 4222 len 36864
26551 091/300 Read block 2565 len 40960
26552 092/300 Write block 1208 len 4096 data: 4
26553 093/300 Write block 4309 len 8192 data: 8 7
26554 094/300 Write block 2611 len 12288 data: 5 6 5
26555 095/300 Write block 3836 len 8192 data: 7 6
26556 096/300 Write block 4380 len 40960 data: 3 4 6 5 6 6 5 5 5 5
26557 097/300 Read block 972 len 8192
26558 098/300 Write block 19 len 4096 data: 4
26559 099/300 Read block 3651 len 36864
26560 100/300 Read block 1056 len 12288
26561 101/300 Write block 3562 len 40960 data: 6 6 5 3 3 6 6 7 5 4
26562 102/300 Write block 2229 len 8192 data: 6 6
26563 103/300 Write block 3553 len 12288 data: 9 7 8
26564 104/300 Read block 3596 len 28672
26565 105/300 Write block 1778 len 40960 data: 6 6 6 4 5 5 5 4 5 5
26566 106/300 Write block 1463 len 28672 data: 3 3 3 3 4 4 3
26567 107/300 Write block 80 len 16384 data: 5 5 6 6
26568 108/300 Read block 4584 len 40960
26569 109/300 Read block 3513 len 24576
26570 110/300 Read block 3467 len 24576
26571 111/300 Read block 1136 len 12288
26572 112/300 Write block 4964 len 12288 data: 6 6 5
26573 113/300 Read block 3020 len 20480
26574 114/300 Write block 2129 len 36864 data: 3 4 4 5 5 5 5 3 4
26575 115/300 Read block 2325 len 28672
26576 116/300 Write block 1974 len 16384 data: 5 3 3 3
26577 117/300 Read block 4788 len 4096
26578 118/300 Flush
26579 119/300 Write block 911 len 8192 data: 4 5
26580 120/300 Write block 1259 len 12288 data: 5 5 5
26581 121/300 Read block 447 len 24576
26582 122/300 Read block 3639 len 16384
26583 123/300 Write block 1432 len 12288 data: 6 7 6
26584 124/300 Read block 2984 len 8192
26585 125/300 Write block 4582 len 28672 data: 3 3 2 2 3 3 4
26586 126/300 Write block 2087 len 20480 data: 3 4 3 3 3
26587 127/300 Write block 4586 len 28672 data: 4 4 5 4 5 6 6
26588 128/300 Write block 4025 len 24576 data: 3 3 3 3 3 3
26589 129/300 Read block 46 len 32768
26590 130/300 Write block 3560 len 16384 data: 6 7 7 7
26591 131/300 Write block 1531 len 8192 data: 4 4
26592 132/300 Write block 17 len 20480 data: 5 4 5 6 6
26593 133/300 Read block 2003 len 8192
26594 134/300 Write block 3688 len 32768 data: 4 3 4 5 7 9 10 10
26595 135/300 Write block 1823 len 32768 data: 5 6 5 6 7 9 7 8
26596 136/300 Read block 4734 len 20480
26597 137/300 Write block 4046 len 24576 data: 5 5 5 4 4 3
26598 138/300 Read block 1819 len 8192
26599 139/300 Flush
26600 140/300 Write block 58 len 24576 data: 4 4 4 4 5 4
26601 141/300 Write block 37 len 24576 data: 3 2 2 2 2 4
26602 142/300 Read block 4816 len 28672
26603 143/300 Write block 1265 len 12288 data: 6 6 6
26604 144/300 Write block 3019 len 24576 data: 5 4 5 6 6 6
26605 145/300 Write block 124 len 16384 data: 5 5 5 4
26606 146/300 Write block 436 len 12288 data: 5 6 8
26607 147/300 Write block 1794 len 40960 data: 7 7 4 4 4 4 4 4 4 5
26608 148/300 Read block 4801 len 28672
26609 149/300 Write block 2426 len 16384 data: 6 6 6 6
26610 150/300 Read block 4032 len 8192
26611 151/300 Write block 3877 len 32768 data: 3 3 4 4 5 7 9 8
26612 152/300 Read block 918 len 16384
26613 153/300 Write block 2050 len 24576 data: 4 4 4 4 4 4
26614 154/300 Read block 2255 len 32768
26615 155/300 Read block 3576 len 28672
26616 156/300 Write block 3334 len 36864 data: 7 7 6 5 6 7 6 6 7
26617 157/300 Write block 2405 len 16384 data: 4 3 3 3
26618 158/300 Write block 1740 len 12288 data: 3 4 5
26619 159/300 Read block 1472 len 20480
26620 160/300 Read block 4255 len 8192
26621 161/300 Flush
26622 162/300 Write block 3955 len 20480 data: 4 5 4 4 4
26623 163/300 Flush
26624 164/300 Read block 3635 len 20480
26625 165/300 Read block 974 len 36864
26626 166/300 Write block 1594 len 36864 data: 6 5 6 8 8 9 9 10 9
26627 167/300 Write block 564 len 4096 data: 5
26628 168/300 Read block 3442 len 4096
26629 169/300 Read block 4434 len 28672
26630 170/300 Write block 1826 len 36864 data: 7 8 10 8 9 6 6 4 3
26631 171/300 Write block 1244 len 28672 data: 3 3 3 4 4 5 5
26632 172/300 Write block 2092 len 8192 data: 4 3
26633 173/300 Write block 4526 len 36864 data: 8 8 9 7 8 7 7 6 7
26634 174/300 Read block 2280 len 4096
26635 175/300 Read block 4078 len 36864
26636 176/300 Write block 2338 len 40960 data: 4 5 5 4 5 5 4 4 4 3
26637 177/300 Read block 346 len 40960
26638 178/300 Read block 864 len 16384
26639 179/300 Write block 3041 len 32768 data: 2 2 2 2 2 3 4 5
26640 180/300 Write block 1828 len 40960 data: 11 9 10 7 7 5 4 4 4 5
26641 181/300 Read block 576 len 32768
26642 182/300 Write block 3413 len 32768 data: 5 5 4 4 3 3 2 2
26643 183/300 Read block 1109 len 20480
26644 184/300 Flush
26645 185/300 Write block 4288 len 24576 data: 6 4 4 4 5 5
26646 186/300 Write block 2003 len 4096 data: 3
26647 187/300 Read block 325 len 40960
26648 188/300 Write block 1049 len 40960 data: 8 9 9 8 7 6 6 5 3 3
26649 189/300 Read block 2338 len 40960
26650 190/300 Read block 3997 len 20480
26651 191/300 Write block 3495 len 40960 data: 7 3 3 3 3 3 4 5 6 6
26652 192/300 Flush
26653 193/300 Read block 1848 len 24576
26654 194/300 Write block 1914 len 12288 data: 5 5 7
26655 195/300 Read block 379 len 12288
26656 196/300 Write block 2566 len 4096 data: 5
26657 197/300 Read block 4340 len 36864
26658 198/300 Read block 2125 len 32768
26659 199/300 Read block 2416 len 12288
26660 200/300 Read block 1590 len 40960
26661 201/300 Read block 731 len 16384
26662 202/300 Write block 3952 len 36864 data: 4 4 4 5 6 5 5 5 3
26663 203/300 Read block 2741 len 36864
26664 204/300 Write block 2844 len 8192 data: 2 3
26665 205/300 Read block 3372 len 16384
26666 206/300 Write block 2622 len 24576 data: 4 3 4 3 3 3
26667 207/300 Read block 2004 len 32768
26668 208/300 Write block 1582 len 24576 data: 4 4 4 4 5 6
26669 209/300 Read block 1609 len 16384
26670 210/300 Flush
26671 211/300 Read block 1227 len 36864
26672 212/300 Write block 1863 len 8192 data: 5 5
26673 213/300 Read block 319 len 12288
26674 214/300 Write block 2827 len 24576 data: 8 6 6 5 5 2
26675 215/300 Read block 4777 len 40960
26676 216/300 Read block 4758 len 32768
26677 217/300 Read block 3029 len 16384
26678 218/300 Write block 2382 len 16384 data: 4 4 5 3
26679 219/300 Read block 1347 len 12288
26680 220/300 Flush
26681 221/300 Read block 1164 len 36864
26682 222/300 Flush
26683 223/300 Flush
26684 224/300 Read block 635 len 4096
26685 225/300 Write block 2139 len 8192 data: 5 4
26686 226/300 Write block 1575 len 32768 data: 7 7 5 3 4 6 5 5
26687 227/300 Read block 371 len 12288
26688 228/300 Read block 848 len 36864
26689 229/300 Read block 2661 len 8192
26690 230/300 Write block 563 len 36864 data: 5 6 4 5 5 7 7 5 4
26691 231/300 Read block 519 len 4096
26692 232/300 Read block 2016 len 16384
26693 233/300 Flush
26694 234/300 Read block 315 len 32768
26695 235/300 Read block 3162 len 12288
26696 236/300 Write block 2090 len 36864 data: 4 4 5 4 3 2 2 3 3
26697 237/300 Write block 562 len 12288 data: 4 6 7
26698 238/300 Flush
26699 239/300 Flush
26700 240/300 Read block 2801 len 8192
26701 241/300 Write block 1468 len 36864 data: 5 4 3 3 3 4 4 5 4
26702 242/300 Read block 3516 len 32768
26703 243/300 Read block 4732 len 20480
26704 244/300 Write block 4969 len 24576 data: 5 5 4 3 4 4
26705 245/300 Write block 1619 len 8192 data: 4 5
26706 246/300 Flush
26707 247/300 Read block 4872 len 28672
26708 248/300 Flush
26709 249/300 Read block 2161 len 12288
26710 250/300 Write block 933 len 32768 data: 3 3 3 3 3 3 4 4
26711 251/300 Flush
26712 252/300 Write block 2749 len 24576 data: 3 4 3 4 5 6
26713 253/300 Write block 2775 len 16384 data: 4 3 4 3
26714 254/300 Write block 2880 len 4096 data: 5
26715 255/300 Read block 4443 len 40960
26716 256/300 Write block 1773 len 36864 data: 6 7 7 5 5 7 7 7 5
26717 257/300 Read block 2849 len 8192
26718 258/300 Read block 3395 len 16384
26719 259/300 Write block 2774 len 20480 data: 4 5 4 5 4
26720 260/300 Write block 2350 len 8192 data: 5 5
26721 261/300 Read block 1909 len 40960
26722 262/300 Flush
26723 263/300 Read block 1301 len 40960
26724 264/300 Read block 4719 len 8192
26725 265/300 Read block 4475 len 8192
26726 266/300 Read block 4286 len 36864
26727 267/300 Read block 4452 len 4096
26728 268/300 Read block 781 len 16384
26729 269/300 Write block 2956 len 28672 data: 3 4 5 5 5 4 5
26730 270/300 Read block 962 len 40960
26731 271/300 Read block 1539 len 28672
26732 272/300 Write block 3085 len 36864 data: 9 6 5 6 6 5 5 5 5
26733 273/300 Write block 2769 len 4096 data: 4
26734 274/300 Read block 3298 len 4096
26735 275/300 Write block 3892 len 12288 data: 5 4 4
26736 276/300 Flush
26737 277/300 Write block 3143 len 40960 data: 5 4 4 4 4 5 5 4 4 4
26738 278/300 Read block 4753 len 12288
26739 279/300 Read block 1120 len 36864
26740 280/300 Read block 4585 len 36864
26741 281/300 Read block 4942 len 16384
26742 282/300 Read block 2521 len 40960
26743 283/300 Write block 625 len 36864 data: 5 6 5 4 4 4 5 6 4
26744 284/300 Read block 3726 len 20480
26745 285/300 Write block 668 len 16384 data: 3 3 4 3
26746 286/300 Read block 3390 len 32768
26747 287/300 Read block 2448 len 36864
26748 288/300 Write block 3148 len 4096 data: 6
26749 289/300 Write block 4056 len 32768 data: 2 2 3 3 5 6 6 5
26750 290/300 Read block 3393 len 8192
26751 291/300 Read block 433 len 24576
26752 292/300 Read block 2517 len 12288
26753 293/300 Read block 40 len 24576
26754 294/300 Read block 2106 len 16384
26755 295/300 Read block 4857 len 24576
26756 296/300 Read block 4748 len 36864
26757 297/300 Flush
26758 298/300 Read block 483 len 20480
26759 299/300 Write block 2240 len 36864 data: 5 7 5 5 5 5 4 4 5
26760 300/300 Read block 4462 len 32768
26761 301/300 Write block 871 len 12288 data: 5 6 6
26762 Main task has work to do, go find it
26763 got dsc Start(1)
26764 start 1
26765 Nov 07 14:08:56.905 INFO request completed, latency_us: 217, response_code: 204, uri: /start/cid/1, method: POST, req_id: ed0bdb90-7c5a-497d-8634-b8e11d02c897, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
26766 Replay: started 1, returned:Ok(())
26767 ----------------------------------------------------------------
26768 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26769 States: Active Offline Active
26770 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26771 7955 Acked Read 10 Done Sent Done false
26772 7956 Acked Write 3 Done Sent Done false
26773 7957 Acked Write 7 Done Sent Done false
26774 7958 Acked Write 3 Done Sent Done false
26775 7959 Acked Read 3 Done Sent Done false
26776 7960 Acked Write 1 Done Sent Done false
26777 7961 Acked Read 1 Done Sent Done false
26778 7962 Acked Read 1 Done Sent Done false
26779 7963 Acked Write 5 Done Sent Done false
26780 7964 Acked Read 9 Done Sent Done false
26781 7965 Acked Read 1 Done Sent Done false
26782 7966 Acked Read 10 Done Sent Done false
26783 7967 Acked Flush 0 Done Sent Done false
26784 7968 Acked Read 6 Done Sent Done false
26785 7969 Acked Read 5 Done Sent Done false
26786 7970 Acked Read 4 Done Sent Done false
26787 7971 Acked Read 7 Done Sent Done false
26788 7972 Acked Write 6 Done Sent Done false
26789 7973 Acked Read 4 Done Sent Done false
26790 7974 Acked Write 5 Done Sent Done false
26791 7975 Acked Write 6 Done Sent Done false
26792 7976 Acked Write 1 Done Sent Done false
26793 7977 Acked Read 2 Done Sent Done false
26794 7978 Acked Read 10 Done Sent Done false
26795 7979 Acked Read 9 Done Sent Done false
26796 7980 Acked Write 5 Done Sent Done false
26797 7981 Acked Read 1 Done Sent Done false
26798 7982 Acked Read 7 Done Sent Done false
26799 7983 Acked Read 2 Done Sent Done false
26800 7984 Acked Read 5 Done Sent Done false
26801 7985 Acked Flush 0 Done Sent Done false
26802 7986 Acked Read 6 Done Sent Done false
26803 7987 Acked Write 1 Done Sent Done false
26804 7988 Acked Read 4 Done Sent Done false
26805 7989 Acked Read 6 Done Sent Done false
26806 7990 Acked Read 9 Done Sent Done false
26807 7991 Acked Read 7 Done Sent Done false
26808 7992 Acked Write 7 Done Sent Done false
26809 7993 Acked Write 6 Done Sent Done false
26810 7994 Acked Write 3 Done Sent Done false
26811 7995 Acked Write 1 Done Sent Done false
26812 7996 Acked Read 8 Done Sent Done false
26813 7997 Acked Read 1 Done Sent Done false
26814 7998 Acked Read 2 Done Sent Done false
26815 7999 Acked Read 4 Done Sent Done false
26816 8000 Acked Read 6 Done Sent Done false
26817 8001 Acked Read 1 Done Sent Done false
26818 8002 Acked Read 9 Done Sent Done false
26819 8003 Acked Flush 0 Done Sent Done false
26820 8004 Acked Write 3 Done Sent Done false
26821 8005 Acked Write 7 Done Sent Done false
26822 8006 Acked Read 7 Done Sent Done false
26823 8007 Acked Flush 0 Done Sent Done false
26824 8008 Acked Read 8 Done Sent Done false
26825 8009 Acked Flush 0 Done Sent Done false
26826 8010 Acked Write 2 Done Sent Done false
26827 8011 Acked Write 10 Done Sent Done false
26828 8012 Acked Read 7 Done Sent Done false
26829 8013 Acked Write 2 Done Sent Done false
26830 8014 Acked Write 7 Done Sent Done false
26831 8015 Acked Read 5 Done Sent Done false
26832 8016 Acked Read 3 Done Sent Done false
26833 8017 Acked Flush 0 Done Sent Done false
26834 8018 Acked Read 1 Done Sent Done false
26835 8019 Acked Flush 0 Done Sent Done false
26836 8020 Acked Read 10 Done Sent Done false
26837 8021 Acked Read 6 Done Sent Done false
26838 8022 Acked Read 9 Done Sent Done false
26839 8023 Acked Write 4 Done Sent Done false
26840 8024 Acked Read 3 Done Sent Done false
26841 8025 Acked Write 6 Done Sent Done false
26842 8026 Acked Read 7 Done Sent Done false
26843 8027 Acked Read 10 Done Sent Done false
26844 8028 Acked Read 10 Done Sent Done false
26845 8029 Acked Write 10 Done Sent Done false
26846 8030 Acked Read 6 Done Sent Done false
26847 8031 Acked Write 9 Done Sent Done false
26848 8032 Acked Read 8 Done Sent Done false
26849 8033 Acked Read 4 Done Sent Done false
26850 8034 Acked Read 10 Done Sent Done false
26851 8035 Acked Read 3 Done Sent Done false
26852 8036 Acked Read 5 Done Sent Done false
26853 8037 Acked Flush 0 Done Sent Done false
26854 8038 Acked Read 9 Done Sent Done false
26855 8039 Acked Read 8 Done Sent Done false
26856 8040 Acked Write 10 Done Sent Done false
26857 8041 Acked Write 5 Done Sent Done false
26858 8042 Acked Read 7 Done Sent Done false
26859 8043 Acked Read 3 Done Sent Done false
26860 8044 Acked Read 9 Done Sent Done false
26861 8045 Acked Read 10 Done Sent Done false
26862 8046 Acked Write 1 Done Sent Done false
26863 8047 Acked Write 2 Done Sent Done false
26864 8048 Acked Write 3 Done Sent Done false
26865 8049 Acked Write 2 Done Sent Done false
26866 8050 Acked Write 10 Done Sent Done false
26867 8051 Acked Read 2 Done Sent Done false
26868 8052 Acked Write 1 Done Sent Done false
26869 8053 Acked Read 9 Done Sent Done false
26870 8054 Acked Read 3 Done Sent Done false
26871 8055 Acked Write 10 Done Sent Done false
26872 8056 Acked Write 2 Done Sent Done false
26873 8057 Acked Write 3 Done Sent Done false
26874 8058 Acked Read 7 Done Sent Done false
26875 8059 Acked Write 10 Done Sent Done false
26876 8060 Acked Write 7 Done Sent Done false
26877 8061 Acked Write 4 Done Sent Done false
26878 8062 Acked Read 10 Done Sent Done false
26879 8063 Acked Read 6 Done Sent Done false
26880 8064 Acked Read 6 Done Sent Done false
26881 8065 Acked Read 3 Done Sent Done false
26882 8066 Acked Write 3 Done Sent Done false
26883 8067 Acked Read 5 Done Sent Done false
26884 8068 Acked Write 9 Done Sent Done false
26885 8069 Acked Read 7 Done Sent Done false
26886 8070 Acked Write 4 Done Sent Done false
26887 8071 Acked Read 1 Done Sent Done false
26888 8072 Acked Flush 0 Done Sent Done false
26889 8073 Acked Write 2 Done Sent Done false
26890 8074 Acked Write 3 Done Sent Done false
26891 8075 Acked Read 6 Done Sent Done false
26892 8076 Acked Read 4 Done Sent Done false
26893 8077 Acked Write 3 Done Sent Done false
26894 8078 Acked Read 2 Done Sent Done false
26895 8079 Acked Write 7 Done Sent Done false
26896 8080 Acked Write 5 Done Sent Done false
26897 8081 Acked Write 7 Done Sent Done false
26898 8082 Acked Write 6 Done Sent Done false
26899 8083 Acked Read 8 Done Sent Done false
26900 8084 Acked Write 4 Done Sent Done false
26901 8085 Acked Write 2 Done Sent Done false
26902 8086 Acked Write 5 Done Sent Done false
26903 8087 Acked Read 2 Done Sent Done false
26904 8088 Acked Write 8 Done Sent Done false
26905 8089 Acked Write 8 Done Sent Done false
26906 8090 Acked Read 5 Done Sent Done false
26907 8091 Acked Write 6 Done Sent Done false
26908 8092 Acked Read 2 Done Sent Done false
26909 8093 Acked Flush 0 Done Sent Done false
26910 8094 Acked Write 6 Done Sent Done false
26911 8095 Acked Write 6 Done Sent Done false
26912 8096 Acked Read 7 Done Sent Done false
26913 8097 Acked Write 3 Done Sent Done false
26914 8098 Acked Write 6 Done Sent Done false
26915 8099 Acked Write 4 Done Sent Done false
26916 8100 Acked Write 3 Done Sent Done false
26917 8101 Acked Write 10 Done Sent Done false
26918 8102 Acked Read 7 Done Sent Done false
26919 8103 Acked Write 4 Done Sent Done false
26920 8104 Acked Read 2 Done Sent Done false
26921 8105 Acked Write 8 Done Sent Done false
26922 8106 Acked Read 4 Done Sent Done false
26923 8107 Acked Write 6 Done Sent Done false
26924 8108 Acked Read 8 Done Sent Done false
26925 8109 Acked Read 7 Done Sent Done false
26926 8110 Acked Write 9 Done Sent Done false
26927 8111 Acked Write 4 Done Sent Done false
26928 8112 Acked Write 3 Done Sent Done false
26929 8113 Acked Read 5 Done Sent Done false
26930 8114 Acked Read 2 Done Sent Done false
26931 8115 Acked Flush 0 Done Sent Done false
26932 8116 Acked Write 5 Done Sent Done false
26933 8117 Acked Flush 0 Done Sent Done false
26934 8118 Acked Read 5 Done Sent Done false
26935 8119 Acked Read 9 Done Sent Done false
26936 8120 Acked Write 9 Done Sent Done false
26937 8121 Acked Write 1 Done Sent Done false
26938 8122 Acked Read 1 Done Sent Done false
26939 8123 Acked Read 7 Done Sent Done false
26940 8124 Acked Write 9 Done Sent Done false
26941 8125 Acked Write 7 Done Sent Done false
26942 8126 Acked Write 2 Done Sent Done false
26943 8127 Acked Write 9 Done Sent Done false
26944 8128 Acked Read 1 Done Sent Done false
26945 8129 Acked Read 9 Done Sent Done false
26946 8130 Acked Write 10 Done Sent Done false
26947 8131 Acked Read 10 Done Sent Done false
26948 8132 Acked Read 4 Done Sent Done false
26949 8133 Acked Write 8 Done Sent Done false
26950 8134 Acked Write 10 Done Sent Done false
26951 8135 Acked Read 8 Done Sent Done false
26952 8136 Acked Write 8 Done Sent Done false
26953 8137 Acked Read 5 Done Sent Done false
26954 8138 Acked Flush 0 Done Sent Done false
26955 8139 Acked Write 6 Done Sent Done false
26956 8140 Acked Write 1 Done Sent Done false
26957 8141 Acked Read 10 Done Sent Done false
26958 8142 Acked Write 10 Done Sent Done false
26959 8143 Acked Read 10 Done Sent Done false
26960 8144 Acked Read 5 Done Sent Done false
26961 8145 Acked Write 10 Done Sent Done false
26962 8146 Acked Flush 0 Done Sent Done false
26963 8147 Acked Read 6 Done Sent Done false
26964 8148 Acked Write 3 Done Sent Done false
26965 8149 Acked Read 3 Done Sent Done false
26966 8150 Acked Write 1 Done Sent Done false
26967 8151 Acked Read 9 Done Sent Done false
26968 8152 Acked Read 8 Done Sent Done false
26969 8153 Acked Read 3 Done Sent Done false
26970 8154 Acked Read 10 Done Sent Done false
26971 8155 Acked Read 4 Done Sent Done false
26972 8156 Acked Write 9 Done Sent Done false
26973 8157 Acked Read 9 Done Sent Done false
26974 8158 Acked Write 2 Done Sent Done false
26975 8159 Acked Read 4 Done Sent Done false
26976 8160 Acked Write 6 Done Sent Done false
26977 8161 Acked Read 8 Done Sent Done false
26978 8162 Acked Write 6 Done Sent Done false
26979 8163 Acked Read 4 Done Sent Done false
26980 8164 Acked Flush 0 Done Sent Done false
26981 8165 Acked Read 9 Done Sent Done false
26982 8166 Acked Write 2 Done Sent Done false
26983 8167 Acked Read 3 Done Sent Done false
26984 8168 Acked Write 6 Done Sent Done false
26985 8169 Acked Read 10 Done Sent Done false
26986 8170 Acked Read 8 Done Sent Done false
26987 8171 Acked Read 4 Done Sent Done false
26988 8172 Acked Write 4 Done Sent Done false
26989 8173 Acked Read 3 Done Sent Done false
26990 8174 Acked Flush 0 Done Sent Done false
26991 8175 Acked Read 9 Done Sent Done false
26992 8176 Acked Flush 0 Done Sent Done false
26993 8177 Acked Flush 0 Done Sent Done false
26994 8178 Acked Read 1 Done Sent Done false
26995 8179 Acked Write 2 Done Sent Done false
26996 8180 Acked Write 8 Done Sent Done false
26997 8181 Acked Read 3 Done Sent Done false
26998 8182 Acked Read 9 Done Sent Done false
26999 8183 Acked Read 2 Done Sent Done false
27000 8184 Acked Write 9 Done Sent Done false
27001 8185 Acked Read 1 Done Sent Done false
27002 8186 Acked Read 4 Done Sent Done false
27003 8187 Acked Flush 0 Done Sent Done false
27004 8188 Acked Read 8 Done Sent Done false
27005 8189 Acked Read 3 Done Sent Done false
27006 8190 Acked Write 9 Done Sent Done false
27007 8191 Acked Write 3 Done Sent Done false
27008 8192 Acked Flush 0 Done Sent Done false
27009 8193 Acked Flush 0 Done Sent Done false
27010 8194 Acked Read 2 Done Sent Done false
27011 8195 Acked Write 9 Done Sent Done false
27012 8196 Acked Read 8 Done Sent Done false
27013 8197 Acked Read 5 Done Sent Done false
27014 8198 Acked Write 6 Done Sent Done false
27015 8199 Acked Write 2 Done Sent Done false
27016 8200 Acked Flush 0 Done Sent Done false
27017 8201 Acked Read 7 Done Sent Done false
27018 8202 Acked Flush 0 Done Sent Done false
27019 8203 Acked Read 3 Done Sent Done false
27020 8204 Acked Write 8 Done Sent Done false
27021 8205 Acked Flush 0 Done Sent Done false
27022 8206 Acked Write 6 Done Sent Done false
27023 8207 Acked Write 4 Done Sent Done false
27024 8208 Acked Write 1 Done Sent Done false
27025 8209 Acked Read 10 Done Sent Done false
27026 8210 Acked Write 9 Done Sent Done false
27027 8211 Acked Read 2 Done Sent Done false
27028 8212 Acked Read 4 Done Sent Done false
27029 8213 Acked Write 5 Done Sent Done false
27030 8214 Acked Write 2 Done Sent Done false
27031 8215 Acked Read 10 Done Sent Done false
27032 8216 Acked Flush 0 Done Sent Done false
27033 8217 Acked Read 10 Done Sent Done false
27034 8218 Acked Read 2 Done Sent Done false
27035 8219 Acked Read 2 Done Sent Done false
27036 8220 Acked Read 9 Done Sent Done false
27037 8221 Acked Read 1 Done Sent Done false
27038 8222 Acked Read 4 Done Sent Done false
27039 8223 Acked Write 7 Done Sent Done false
27040 8224 Acked Read 10 Done Sent Done false
27041 8225 Acked Read 7 Done Sent Done false
27042 8226 Acked Write 9 Done Sent Done false
27043 8227 Acked Write 1 Done Sent Done false
27044 8228 Acked Read 1 Done Sent Done false
27045 8229 Acked Write 3 Done Sent Done false
27046 8230 Acked Flush 0 Done Sent Done false
27047 8231 Acked Write 10 Done Sent Done false
27048 8232 Acked Read 3 Done Sent Done false
27049 8233 Acked Read 9 Done Sent Done false
27050 8234 Acked Read 9 Done Sent Done false
27051 8235 Acked Read 4 Done Sent Done false
27052 8236 Acked Read 10 Done Sent Done false
27053 8237 Acked Write 9 Done Sent Done false
27054 8238 Acked Read 5 Done Sent Done false
27055 8239 Acked Write 4 Done Sent Done false
27056 8240 Acked Read 8 Done Sent Done false
27057 8241 Acked Read 9 Done Sent Done false
27058 8242 Acked Write 1 Done Sent Done false
27059 8243 Acked Write 8 Done Sent Done false
27060 8244 Acked Read 2 Done Sent Done false
27061 8245 Acked Read 6 Done Sent Done false
27062 8246 Acked Read 3 Done Sent Done false
27063 8247 Acked Read 6 Done Sent Done false
27064 8248 Acked Read 4 Done Sent Done false
27065 8249 Acked Read 6 Done Sent Done false
27066 8250 Acked Read 9 Done Sent Done false
27067 8251 Acked Flush 0 Done Sent Done false
27068 8252 Acked Read 5 Done Sent Done false
27069 8253 Acked Write 9 Done Sent Done false
27070 8254 Acked Read 8 Done Sent Done false
27071 8255 Acked Write 3 Done Sent Done false
27072 STATES DS:0 DS:1 DS:2 TOTAL
27073 Sent 0 301 0 301
27074 Done 301 0 301 602
27075 Skipped 0 0 0 0
27076 Error 0 0 0 0
27077 Last Flush: 8251 7954 8251
27078 Downstairs last five completed: 7954 7953 7952 7951 7950
27079 Upstairs last five completed: 8255 8254 8253 8252 8251
27080 CLIENT: Up:0 ds:301 act:2
27081 [8820] got start action
27082 [8820] I am going to restart
27083 Starting downstairs at port 8820
27084 Make output file at "/tmp/dsc/downstairs-8820.txt"
27085 [8820][1] reports Starting
27086 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1223)
27087 [8820][1] reports Running
27088 ----------------------------------------------------------------
27089 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27090 States: Active Offline Active
27091 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27092 7955 Acked Read 10 Done Sent Done false
27093 7956 Acked Write 3 Done Sent Done false
27094 7957 Acked Write 7 Done Sent Done false
27095 7958 Acked Write 3 Done Sent Done false
27096 7959 Acked Read 3 Done Sent Done false
27097 7960 Acked Write 1 Done Sent Done false
27098 7961 Acked Read 1 Done Sent Done false
27099 7962 Acked Read 1 Done Sent Done false
27100 7963 Acked Write 5 Done Sent Done false
27101 7964 Acked Read 9 Done Sent Done false
27102 7965 Acked Read 1 Done Sent Done false
27103 7966 Acked Read 10 Done Sent Done false
27104 7967 Acked Flush 0 Done Sent Done false
27105 7968 Acked Read 6 Done Sent Done false
27106 7969 Acked Read 5 Done Sent Done false
27107 7970 Acked Read 4 Done Sent Done false
27108 7971 Acked Read 7 Done Sent Done false
27109 7972 Acked Write 6 Done Sent Done false
27110 7973 Acked Read 4 Done Sent Done false
27111 7974 Acked Write 5 Done Sent Done false
27112 7975 Acked Write 6 Done Sent Done false
27113 7976 Acked Write 1 Done Sent Done false
27114 7977 Acked Read 2 Done Sent Done false
27115 7978 Acked Read 10 Done Sent Done false
27116 7979 Acked Read 9 Done Sent Done false
27117 7980 Acked Write 5 Done Sent Done false
27118 7981 Acked Read 1 Done Sent Done false
27119 7982 Acked Read 7 Done Sent Done false
27120 7983 Acked Read 2 Done Sent Done false
27121 7984 Acked Read 5 Done Sent Done false
27122 7985 Acked Flush 0 Done Sent Done false
27123 7986 Acked Read 6 Done Sent Done false
27124 7987 Acked Write 1 Done Sent Done false
27125 7988 Acked Read 4 Done Sent Done false
27126 7989 Acked Read 6 Done Sent Done false
27127 7990 Acked Read 9 Done Sent Done false
27128 7991 Acked Read 7 Done Sent Done false
27129 7992 Acked Write 7 Done Sent Done false
27130 7993 Acked Write 6 Done Sent Done false
27131 7994 Acked Write 3 Done Sent Done false
27132 7995 Acked Write 1 Done Sent Done false
27133 7996 Acked Read 8 Done Sent Done false
27134 7997 Acked Read 1 Done Sent Done false
27135 7998 Acked Read 2 Done Sent Done false
27136 7999 Acked Read 4 Done Sent Done false
27137 8000 Acked Read 6 Done Sent Done false
27138 8001 Acked Read 1 Done Sent Done false
27139 8002 Acked Read 9 Done Sent Done false
27140 8003 Acked Flush 0 Done Sent Done false
27141 8004 Acked Write 3 Done Sent Done false
27142 8005 Acked Write 7 Done Sent Done false
27143 8006 Acked Read 7 Done Sent Done false
27144 8007 Acked Flush 0 Done Sent Done false
27145 8008 Acked Read 8 Done Sent Done false
27146 8009 Acked Flush 0 Done Sent Done false
27147 8010 Acked Write 2 Done Sent Done false
27148 8011 Acked Write 10 Done Sent Done false
27149 8012 Acked Read 7 Done Sent Done false
27150 8013 Acked Write 2 Done Sent Done false
27151 8014 Acked Write 7 Done Sent Done false
27152 8015 Acked Read 5 Done Sent Done false
27153 8016 Acked Read 3 Done Sent Done false
27154 8017 Acked Flush 0 Done Sent Done false
27155 8018 Acked Read 1 Done Sent Done false
27156 8019 Acked Flush 0 Done Sent Done false
27157 8020 Acked Read 10 Done Sent Done false
27158 8021 Acked Read 6 Done Sent Done false
27159 8022 Acked Read 9 Done Sent Done false
27160 8023 Acked Write 4 Done Sent Done false
27161 8024 Acked Read 3 Done Sent Done false
27162 8025 Acked Write 6 Done Sent Done false
27163 8026 Acked Read 7 Done Sent Done false
27164 8027 Acked Read 10 Done Sent Done false
27165 8028 Acked Read 10 Done Sent Done false
27166 8029 Acked Write 10 Done Sent Done false
27167 8030 Acked Read 6 Done Sent Done false
27168 8031 Acked Write 9 Done Sent Done false
27169 8032 Acked Read 8 Done Sent Done false
27170 8033 Acked Read 4 Done Sent Done false
27171 8034 Acked Read 10 Done Sent Done false
27172 8035 Acked Read 3 Done Sent Done false
27173 8036 Acked Read 5 Done Sent Done false
27174 8037 Acked Flush 0 Done Sent Done false
27175 8038 Acked Read 9 Done Sent Done false
27176 8039 Acked Read 8 Done Sent Done false
27177 8040 Acked Write 10 Done Sent Done false
27178 8041 Acked Write 5 Done Sent Done false
27179 8042 Acked Read 7 Done Sent Done false
27180 8043 Acked Read 3 Done Sent Done false
27181 8044 Acked Read 9 Done Sent Done false
27182 8045 Acked Read 10 Done Sent Done false
27183 8046 Acked Write 1 Done Sent Done false
27184 8047 Acked Write 2 Done Sent Done false
27185 8048 Acked Write 3 Done Sent Done false
27186 8049 Acked Write 2 Done Sent Done false
27187 8050 Acked Write 10 Done Sent Done false
27188 8051 Acked Read 2 Done Sent Done false
27189 8052 Acked Write 1 Done Sent Done false
27190 8053 Acked Read 9 Done Sent Done false
27191 8054 Acked Read 3 Done Sent Done false
27192 8055 Acked Write 10 Done Sent Done false
27193 8056 Acked Write 2 Done Sent Done false
27194 8057 Acked Write 3 Done Sent Done false
27195 8058 Acked Read 7 Done Sent Done false
27196 8059 Acked Write 10 Done Sent Done false
27197 8060 Acked Write 7 Done Sent Done false
27198 8061 Acked Write 4 Done Sent Done false
27199 8062 Acked Read 10 Done Sent Done false
27200 8063 Acked Read 6 Done Sent Done false
27201 8064 Acked Read 6 Done Sent Done false
27202 8065 Acked Read 3 Done Sent Done false
27203 8066 Acked Write 3 Done Sent Done false
27204 8067 Acked Read 5 Done Sent Done false
27205 8068 Acked Write 9 Done Sent Done false
27206 8069 Acked Read 7 Done Sent Done false
27207 8070 Acked Write 4 Done Sent Done false
27208 8071 Acked Read 1 Done Sent Done false
27209 8072 Acked Flush 0 Done Sent Done false
27210 8073 Acked Write 2 Done Sent Done false
27211 8074 Acked Write 3 Done Sent Done false
27212 8075 Acked Read 6 Done Sent Done false
27213 8076 Acked Read 4 Done Sent Done false
27214 8077 Acked Write 3 Done Sent Done false
27215 8078 Acked Read 2 Done Sent Done false
27216 8079 Acked Write 7 Done Sent Done false
27217 8080 Acked Write 5 Done Sent Done false
27218 8081 Acked Write 7 Done Sent Done false
27219 8082 Acked Write 6 Done Sent Done false
27220 8083 Acked Read 8 Done Sent Done false
27221 8084 Acked Write 4 Done Sent Done false
27222 8085 Acked Write 2 Done Sent Done false
27223 8086 Acked Write 5 Done Sent Done false
27224 8087 Acked Read 2 Done Sent Done false
27225 8088 Acked Write 8 Done Sent Done false
27226 8089 Acked Write 8 Done Sent Done false
27227 8090 Acked Read 5 Done Sent Done false
27228 8091 Acked Write 6 Done Sent Done false
27229 8092 Acked Read 2 Done Sent Done false
27230 8093 Acked Flush 0 Done Sent Done false
27231 8094 Acked Write 6 Done Sent Done false
27232 8095 Acked Write 6 Done Sent Done false
27233 8096 Acked Read 7 Done Sent Done false
27234 8097 Acked Write 3 Done Sent Done false
27235 8098 Acked Write 6 Done Sent Done false
27236 8099 Acked Write 4 Done Sent Done false
27237 8100 Acked Write 3 Done Sent Done false
27238 8101 Acked Write 10 Done Sent Done false
27239 8102 Acked Read 7 Done Sent Done false
27240 8103 Acked Write 4 Done Sent Done false
27241 8104 Acked Read 2 Done Sent Done false
27242 8105 Acked Write 8 Done Sent Done false
27243 8106 Acked Read 4 Done Sent Done false
27244 8107 Acked Write 6 Done Sent Done false
27245 8108 Acked Read 8 Done Sent Done false
27246 8109 Acked Read 7 Done Sent Done false
27247 8110 Acked Write 9 Done Sent Done false
27248 8111 Acked Write 4 Done Sent Done false
27249 8112 Acked Write 3 Done Sent Done false
27250 8113 Acked Read 5 Done Sent Done false
27251 8114 Acked Read 2 Done Sent Done false
27252 8115 Acked Flush 0 Done Sent Done false
27253 8116 Acked Write 5 Done Sent Done false
27254 8117 Acked Flush 0 Done Sent Done false
27255 8118 Acked Read 5 Done Sent Done false
27256 8119 Acked Read 9 Done Sent Done false
27257 8120 Acked Write 9 Done Sent Done false
27258 8121 Acked Write 1 Done Sent Done false
27259 8122 Acked Read 1 Done Sent Done false
27260 8123 Acked Read 7 Done Sent Done false
27261 8124 Acked Write 9 Done Sent Done false
27262 8125 Acked Write 7 Done Sent Done false
27263 8126 Acked Write 2 Done Sent Done false
27264 8127 Acked Write 9 Done Sent Done false
27265 8128 Acked Read 1 Done Sent Done false
27266 8129 Acked Read 9 Done Sent Done false
27267 8130 Acked Write 10 Done Sent Done false
27268 8131 Acked Read 10 Done Sent Done false
27269 8132 Acked Read 4 Done Sent Done false
27270 8133 Acked Write 8 Done Sent Done false
27271 8134 Acked Write 10 Done Sent Done false
27272 8135 Acked Read 8 Done Sent Done false
27273 8136 Acked Write 8 Done Sent Done false
27274 8137 Acked Read 5 Done Sent Done false
27275 8138 Acked Flush 0 Done Sent Done false
27276 8139 Acked Write 6 Done Sent Done false
27277 8140 Acked Write 1 Done Sent Done false
27278 8141 Acked Read 10 Done Sent Done false
27279 8142 Acked Write 10 Done Sent Done false
27280 8143 Acked Read 10 Done Sent Done false
27281 8144 Acked Read 5 Done Sent Done false
27282 8145 Acked Write 10 Done Sent Done false
27283 8146 Acked Flush 0 Done Sent Done false
27284 8147 Acked Read 6 Done Sent Done false
27285 8148 Acked Write 3 Done Sent Done false
27286 8149 Acked Read 3 Done Sent Done false
27287 8150 Acked Write 1 Done Sent Done false
27288 8151 Acked Read 9 Done Sent Done false
27289 8152 Acked Read 8 Done Sent Done false
27290 8153 Acked Read 3 Done Sent Done false
27291 8154 Acked Read 10 Done Sent Done false
27292 8155 Acked Read 4 Done Sent Done false
27293 8156 Acked Write 9 Done Sent Done false
27294 8157 Acked Read 9 Done Sent Done false
27295 8158 Acked Write 2 Done Sent Done false
27296 8159 Acked Read 4 Done Sent Done false
27297 8160 Acked Write 6 Done Sent Done false
27298 8161 Acked Read 8 Done Sent Done false
27299 8162 Acked Write 6 Done Sent Done false
27300 8163 Acked Read 4 Done Sent Done false
27301 8164 Acked Flush 0 Done Sent Done false
27302 8165 Acked Read 9 Done Sent Done false
27303 8166 Acked Write 2 Done Sent Done false
27304 8167 Acked Read 3 Done Sent Done false
27305 8168 Acked Write 6 Done Sent Done false
27306 8169 Acked Read 10 Done Sent Done false
27307 8170 Acked Read 8 Done Sent Done false
27308 8171 Acked Read 4 Done Sent Done false
27309 8172 Acked Write 4 Done Sent Done false
27310 8173 Acked Read 3 Done Sent Done false
27311 8174 Acked Flush 0 Done Sent Done false
27312 8175 Acked Read 9 Done Sent Done false
27313 8176 Acked Flush 0 Done Sent Done false
27314 8177 Acked Flush 0 Done Sent Done false
27315 8178 Acked Read 1 Done Sent Done false
27316 8179 Acked Write 2 Done Sent Done false
27317 8180 Acked Write 8 Done Sent Done false
27318 8181 Acked Read 3 Done Sent Done false
27319 8182 Acked Read 9 Done Sent Done false
27320 8183 Acked Read 2 Done Sent Done false
27321 8184 Acked Write 9 Done Sent Done false
27322 8185 Acked Read 1 Done Sent Done false
27323 8186 Acked Read 4 Done Sent Done false
27324 8187 Acked Flush 0 Done Sent Done false
27325 8188 Acked Read 8 Done Sent Done false
27326 8189 Acked Read 3 Done Sent Done false
27327 8190 Acked Write 9 Done Sent Done false
27328 8191 Acked Write 3 Done Sent Done false
27329 8192 Acked Flush 0 Done Sent Done false
27330 8193 Acked Flush 0 Done Sent Done false
27331 8194 Acked Read 2 Done Sent Done false
27332 8195 Acked Write 9 Done Sent Done false
27333 8196 Acked Read 8 Done Sent Done false
27334 8197 Acked Read 5 Done Sent Done false
27335 8198 Acked Write 6 Done Sent Done false
27336 8199 Acked Write 2 Done Sent Done false
27337 8200 Acked Flush 0 Done Sent Done false
27338 8201 Acked Read 7 Done Sent Done false
27339 8202 Acked Flush 0 Done Sent Done false
27340 8203 Acked Read 3 Done Sent Done false
27341 8204 Acked Write 8 Done Sent Done false
27342 8205 Acked Flush 0 Done Sent Done false
27343 8206 Acked Write 6 Done Sent Done false
27344 8207 Acked Write 4 Done Sent Done false
27345 8208 Acked Write 1 Done Sent Done false
27346 8209 Acked Read 10 Done Sent Done false
27347 8210 Acked Write 9 Done Sent Done false
27348 8211 Acked Read 2 Done Sent Done false
27349 8212 Acked Read 4 Done Sent Done false
27350 8213 Acked Write 5 Done Sent Done false
27351 8214 Acked Write 2 Done Sent Done false
27352 8215 Acked Read 10 Done Sent Done false
27353 8216 Acked Flush 0 Done Sent Done false
27354 8217 Acked Read 10 Done Sent Done false
27355 8218 Acked Read 2 Done Sent Done false
27356 8219 Acked Read 2 Done Sent Done false
27357 8220 Acked Read 9 Done Sent Done false
27358 8221 Acked Read 1 Done Sent Done false
27359 8222 Acked Read 4 Done Sent Done false
27360 8223 Acked Write 7 Done Sent Done false
27361 8224 Acked Read 10 Done Sent Done false
27362 8225 Acked Read 7 Done Sent Done false
27363 8226 Acked Write 9 Done Sent Done false
27364 8227 Acked Write 1 Done Sent Done false
27365 8228 Acked Read 1 Done Sent Done false
27366 8229 Acked Write 3 Done Sent Done false
27367 8230 Acked Flush 0 Done Sent Done false
27368 8231 Acked Write 10 Done Sent Done false
27369 8232 Acked Read 3 Done Sent Done false
27370 8233 Acked Read 9 Done Sent Done false
27371 8234 Acked Read 9 Done Sent Done false
27372 8235 Acked Read 4 Done Sent Done false
27373 8236 Acked Read 10 Done Sent Done false
27374 8237 Acked Write 9 Done Sent Done false
27375 8238 Acked Read 5 Done Sent Done false
27376 8239 Acked Write 4 Done Sent Done false
27377 8240 Acked Read 8 Done Sent Done false
27378 8241 Acked Read 9 Done Sent Done false
27379 8242 Acked Write 1 Done Sent Done false
27380 8243 Acked Write 8 Done Sent Done false
27381 8244 Acked Read 2 Done Sent Done false
27382 8245 Acked Read 6 Done Sent Done false
27383 8246 Acked Read 3 Done Sent Done false
27384 8247 Acked Read 6 Done Sent Done false
27385 8248 Acked Read 4 Done Sent Done false
27386 8249 Acked Read 6 Done Sent Done false
27387 8250 Acked Read 9 Done Sent Done false
27388 8251 Acked Flush 0 Done Sent Done false
27389 8252 Acked Read 5 Done Sent Done false
27390 8253 Acked Write 9 Done Sent Done false
27391 8254 Acked Read 8 Done Sent Done false
27392 8255 Acked Write 3 Done Sent Done false
27393 8256 Acked Flush 0 Done Sent Done false
27394 STATES DS:0 DS:1 DS:2 TOTAL
27395 Sent 0 302 0 302
27396 Done 302 0 302 604
27397 Skipped 0 0 0 0
27398 Error 0 0 0 0
27399 Last Flush: 8256 7954 8256
27400 Downstairs last five completed: 7954 7953 7952 7951 7950
27401 Upstairs last five completed: 8256 8255 8254 8253 8252
27402 CLIENT: Up:0 ds:302 act:2
274032024-11-07T14:09:02.618ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274042024-11-07T14:09:02.618ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55464) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274052024-11-07T14:09:02.619ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274062024-11-07T14:09:02.619ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274072024-11-07T14:09:02.619ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274082024-11-07T14:09:02.619ZINFOcrucible: send last flush ID to this DS: 7954 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274092024-11-07T14:09:02.620ZINFOcrucible: Replied this last flush ID: 7954 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274102024-11-07T14:09:02.620ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274112024-11-07T14:09:02.620ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274122024-11-07T14:09:02.620ZINFOcrucible: [1] client re-new 302 jobs since flush 7954 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274132024-11-07T14:09:02.620ZINFOcrucible: [1] Replayed 302 jobs since flush: 7954 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27414 ----------------------------------------------------------------
27415 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27416 Downstairs last five completed: 8256 8255 8254 8253 8252
27417 Upstairs last five completed: 8256 8255 8254 8253 8252
27418 CLIENT: Up:0 ds:0 act:3
27419 Replay: All jobs finished
27420 Main task has work to do, go find it
27421 got dsc Stop(0)
27422 stop 0
27423 [8810] Got stop action so:false kr:false
27424 Nov 07 14:09:04.911 INFO request completed, latency_us: 351, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 38bd0c59-746d-4db5-8851-7f03221bb918, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
274252024-11-07T14:09:04.911ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274262024-11-07T14:09:04.911ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274272024-11-07T14:09:04.911ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274282024-11-07T14:09:04.911ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274292024-11-07T14:09:04.911ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
274302024-11-07T14:09:04.911ZINFOcrucible: client task is exiting = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
27431 Nov 07 14:09:04.911 INFO request completed, latency_us: 177, response_code: 200, uri: /state/cid/0, method: GET, req_id: eba940f1-1cc5-4130-8520-ecabb1ec90d8, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
27432 [8810] Exited with: ExitStatus(unix_wait_status(9))
27433 [8810][0] reports Exit
27434 Nov 07 14:09:08.913 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/0, method: GET, req_id: 24e3484d-37ed-4d32-8935-4be27a545947, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
27435 001/300 Write block 2445 len 40960 data: 7 8 6 6 6 7 6 6 6 5
27436 002/300 Write block 2994 len 20480 data: 4 4 4 6 4
27437 003/300 Read block 3829 len 16384
27438 004/300 Read block 805 len 40960
27439 005/300 Read block 3131 len 8192
27440 006/300 Read block 1697 len 28672
27441 007/300 Read block 2151 len 4096
27442 008/300 Read block 1486 len 28672
27443 009/300 Write block 1624 len 32768 data: 4 4 5 4 4 3 4 6
27444 010/300 Read block 3335 len 40960
27445 011/300 Write block 2713 len 36864 data: 3 3 4 4 4 3 3 5 6
27446 012/300 Read block 2664 len 8192
27447 013/300 Flush
27448 014/300 Read block 4503 len 24576
27449 015/300 Flush
27450 016/300 Write block 1942 len 4096 data: 7
27451 017/300 Read block 2305 len 16384
27452 018/300 Read block 400 len 28672
27453 019/300 Write block 3428 len 40960 data: 6 6 6 6 5 5 3 2 2 2
27454 020/300 Flush
27455 021/300 Read block 879 len 20480
27456 022/300 Write block 1144 len 20480 data: 6 5 5 3 3
27457 023/300 Write block 4521 len 40960 data: 5 5 5 5 5 9 9 10 8 9
27458 024/300 Write block 2935 len 36864 data: 2 5 6 5 7 6 6 8 8
27459 025/300 Read block 1865 len 4096
27460 026/300 Write block 3638 len 12288 data: 5 5 5
27461 027/300 Read block 2667 len 20480
27462 028/300 Write block 1399 len 4096 data: 6
27463 029/300 Write block 1749 len 12288 data: 6 9 10
27464 030/300 Read block 2330 len 8192
27465 031/300 Read block 3981 len 36864
27466 032/300 Read block 4318 len 36864
27467 033/300 Write block 998 len 20480 data: 4 4 5 4 3
27468 034/300 Read block 4698 len 28672
27469 035/300 Write block 1212 len 4096 data: 5
27470 036/300 Read block 3853 len 20480
27471 037/300 Read block 3583 len 24576
27472 038/300 Write block 3437 len 40960 data: 3 4 6 6 7 7 7 6 6 6
27473 039/300 Read block 1198 len 20480
27474 040/300 Write block 1615 len 40960 data: 4 4 5 4 5 6 5 4 3 5
27475 041/300 Write block 3086 len 32768 data: 7 6 7 7 6 6 6 6
27476 042/300 Read block 2670 len 36864
27477 043/300 Write block 4963 len 24576 data: 6 7 7 6 6 6
27478 044/300 Read block 3976 len 12288
27479 045/300 Read block 753 len 40960
27480 046/300 Read block 2096 len 12288
27481 047/300 Write block 3491 len 32768 data: 5 6 7 7 8 4 4 4
27482 048/300 Flush
27483 049/300 Read block 2766 len 4096
27484 050/300 Write block 3858 len 28672 data: 3 4 6 6 8 9 9
27485 051/300 Write block 1342 len 20480 data: 6 6 5 5 5
27486 052/300 Write block 1930 len 20480 data: 5 5 5 4 6
27487 053/300 Read block 1911 len 8192
27488 054/300 Read block 2841 len 12288
27489 055/300 Write block 2732 len 28672 data: 8 7 6 4 4 5 4
27490 056/300 Read block 981 len 20480
27491 057/300 Read block 4359 len 12288
27492 058/300 Write block 1026 len 28672 data: 5 5 4 4 3 3 2
27493 059/300 Read block 3699 len 20480
27494 060/300 Write block 1461 len 20480 data: 4 3 4 4 4
27495 061/300 Read block 2385 len 20480
27496 062/300 Flush
27497 063/300 Write block 845 len 36864 data: 2 2 3 4 4 4 4 4 3
27498 064/300 Read block 4002 len 28672
27499 065/300 Read block 2277 len 32768
27500 066/300 Write block 2860 len 32768 data: 4 4 4 4 3 3 4 4
27501 067/300 Flush
27502 068/300 Read block 2329 len 16384
27503 069/300 Flush
27504 070/300 Write block 1335 len 8192 data: 9 8
27505 071/300 Write block 556 len 40960 data: 4 6 5 5 5 5 5 7 8 5
27506 072/300 Write block 4093 len 16384 data: 3 2 4 4
27507 073/300 Read block 2677 len 16384
27508 074/300 Read block 3225 len 20480
27509 075/300 Write block 1680 len 16384 data: 6 7 6 4
27510 076/300 Write block 4983 len 40960 data: 4 4 4 4 4 5 6 6 6 5
27511 077/300 Write block 3847 len 40960 data: 5 4 3 3 3 2 2 2 2 2
27512 078/300 Write block 428 len 36864 data: 5 5 4 4 5 4 3 4 6
27513 079/300 Read block 459 len 16384
27514 080/300 Write block 3920 len 32768 data: 4 5 5 7 8 7 7 8
27515 081/300 Write block 1950 len 24576 data: 10 8 7 7 7 7
27516 082/300 Read block 230 len 8192
27517 083/300 Read block 561 len 4096
27518 084/300 Write block 1900 len 20480 data: 5 6 6 7 6
27519 085/300 Read block 2339 len 28672
27520 086/300 Read block 2420 len 12288
27521 087/300 Read block 282 len 12288
27522 088/300 Read block 4179 len 36864
27523 089/300 Write block 3490 len 12288 data: 5 6 7
27524 090/300 Read block 1667 len 24576
27525 091/300 Read block 4162 len 40960
27526 092/300 Write block 698 len 8192 data: 3 3
27527 093/300 Read block 3964 len 4096
27528 094/300 Read block 4369 len 8192
27529 095/300 Read block 4905 len 8192
27530 096/300 Read block 2061 len 20480
27531 097/300 Write block 4868 len 24576 data: 4 3 5 3 4 4
27532 098/300 Write block 1493 len 16384 data: 7 8 6 6
27533 099/300 Read block 72 len 28672
27534 100/300 Read block 2214 len 16384
27535 101/300 Write block 4251 len 40960 data: 7 5 5 5 5 5 5 6 5 6
27536 102/300 Flush
27537 103/300 Write block 4592 len 12288 data: 7 6 7
27538 104/300 Write block 4661 len 32768 data: 6 6 5 4 4 4 3 4
27539 105/300 Flush
27540 106/300 Write block 427 len 20480 data: 3 6 6 5 5
27541 107/300 Write block 3134 len 40960 data: 6 7 7 4 4 3 4 4 5 6
27542 108/300 Read block 1130 len 40960
27543 109/300 Write block 757 len 28672 data: 4 4 4 5 4 6 5
27544 110/300 Read block 4327 len 36864
27545 111/300 Write block 1405 len 36864 data: 6 6 6 5 4 4 2 3 4
27546 112/300 Read block 1563 len 16384
27547 113/300 Read block 740 len 12288
27548 114/300 Read block 130 len 8192
27549 115/300 Read block 2764 len 20480
27550 116/300 Read block 279 len 24576
27551 117/300 Read block 4665 len 36864
27552 118/300 Write block 3371 len 28672 data: 2 2 5 5 7 9 10
27553 119/300 Read block 3743 len 12288
27554 120/300 Read block 2722 len 40960
27555 121/300 Read block 236 len 24576
27556 122/300 Read block 4194 len 8192
27557 123/300 Write block 4807 len 12288 data: 3 2 3
27558 124/300 Read block 4347 len 24576
27559 125/300 Write block 1188 len 28672 data: 7 7 5 6 6 6 6
27560 126/300 Read block 2641 len 32768
27561 127/300 Write block 3719 len 28672 data: 11 11 10 11 9 7 6
27562 128/300 Write block 4618 len 40960 data: 3 5 6 4 4 3 4 4 4 3
27563 129/300 Read block 1202 len 36864
27564 130/300 Write block 3713 len 32768 data: 7 7 8 9 10 11 12 12
27565 131/300 Read block 4184 len 28672
27566 132/300 Read block 4121 len 32768
27567 133/300 Read block 521 len 36864
27568 134/300 Write block 1392 len 40960 data: 3 4 4 4 5 6 6 7 6 6
27569 135/300 Read block 1657 len 40960
27570 136/300 Read block 206 len 24576
27571 137/300 Write block 7 len 12288 data: 5 5 4
27572 138/300 Write block 260 len 16384 data: 6 8 8 9
27573 139/300 Write block 4922 len 12288 data: 6 5 6
27574 140/300 Flush
27575 141/300 Write block 426 len 20480 data: 4 4 7 7 6
27576 142/300 Read block 1170 len 16384
27577 143/300 Read block 3000 len 12288
27578 144/300 Read block 2283 len 12288
27579 145/300 Write block 2250 len 16384 data: 5 6 6 4
27580 146/300 Read block 3266 len 12288
27581 147/300 Read block 3427 len 32768
27582 148/300 Write block 4944 len 36864 data: 2 3 3 3 4 5 6 6 7
27583 149/300 Read block 3664 len 24576
27584 150/300 Read block 1783 len 36864
27585 151/300 Read block 4718 len 8192
27586 152/300 Read block 1726 len 36864
27587 153/300 Read block 1259 len 32768
27588 154/300 Write block 181 len 36864 data: 6 5 3 4 4 6 5 5 6
27589 155/300 Read block 4274 len 24576
27590 156/300 Read block 1893 len 12288
27591 157/300 Read block 4888 len 16384
27592 158/300 Flush
27593 159/300 Read block 3170 len 40960
27594 160/300 Write block 2767 len 24576 data: 2 3 5 4 3 3
27595 161/300 Read block 1656 len 24576
27596 162/300 Write block 4879 len 32768 data: 2 3 3 4 4 4 4 3
27597 163/300 Read block 4037 len 12288
27598 164/300 Read block 780 len 24576
27599 165/300 Read block 1468 len 8192
27600 166/300 Read block 3222 len 16384
27601 167/300 Write block 1337 len 24576 data: 8 7 5 5 5 7
27602 168/300 Read block 408 len 20480
27603 169/300 Write block 90 len 32768 data: 3 4 4 4 3 3 3 3
27604 170/300 Write block 340 len 36864 data: 3 3 6 5 7 8 6 6 4
27605 171/300 Read block 4933 len 8192
27606 172/300 Flush
27607 173/300 Flush
27608 174/300 Read block 905 len 16384
27609 175/300 Write block 3945 len 24576 data: 2 2 2 2 2 3
27610 176/300 Write block 3020 len 12288 data: 5 6 7
27611 177/300 Read block 2321 len 36864
27612 178/300 Read block 4719 len 32768
27613 179/300 Write block 1140 len 32768 data: 5 6 6 6 7 6 6 4
27614 180/300 Write block 3415 len 28672 data: 5 5 4 4 3 3 2
27615 181/300 Write block 3187 len 4096 data: 4
27616 182/300 Write block 1911 len 4096 data: 3
27617 183/300 Write block 4087 len 20480 data: 4 4 4 4 4
27618 184/300 Read block 2221 len 36864
27619 185/300 Read block 4949 len 28672
27620 186/300 Read block 4131 len 20480
27621 187/300 Write block 4690 len 36864 data: 4 3 3 4 5 5 6 9 7
27622 188/300 Read block 3931 len 4096
27623 189/300 Read block 4264 len 20480
27624 190/300 Write block 4026 len 36864 data: 4 4 4 4 4 2 2 3 4
27625 191/300 Write block 4525 len 40960 data: 6 10 10 11 9 10 8 8 7 8
27626 192/300 Read block 3414 len 12288
27627 193/300 Read block 1681 len 20480
27628 194/300 Read block 3049 len 16384
27629 195/300 Read block 4021 len 32768
27630 196/300 Read block 4286 len 12288
27631 197/300 Write block 2695 len 4096 data: 5
27632 198/300 Read block 2769 len 32768
27633 199/300 Read block 2385 len 16384
27634 200/300 Read block 3357 len 8192
27635 201/300 Read block 3473 len 36864
27636 202/300 Read block 4224 len 4096
27637 203/300 Write block 182 len 36864 data: 6 4 5 5 7 6 6 7 7
27638 204/300 Read block 1174 len 32768
27639 205/300 Write block 4906 len 16384 data: 5 5 6 6
27640 206/300 Write block 2813 len 36864 data: 4 4 4 5 7 6 6 6 6
27641 207/300 Write block 627 len 20480 data: 6 5 5 5 6
27642 208/300 Read block 2841 len 32768
27643 209/300 Flush
27644 210/300 Write block 2967 len 40960 data: 5 4 6 6 8 8 8 8 7 8
27645 211/300 Read block 4668 len 28672
27646 212/300 Read block 4293 len 36864
27647 213/300 Read block 192 len 28672
27648 214/300 Read block 1841 len 16384
27649 215/300 Read block 1463 len 40960
27650 216/300 Write block 2447 len 24576 data: 7 7 7 8 7 7
27651 217/300 Write block 4411 len 8192 data: 5 4
27652 218/300 Flush
27653 219/300 Read block 3942 len 36864
27654 220/300 Write block 111 len 8192 data: 3 3
27655 221/300 Read block 2598 len 24576
27656 222/300 Write block 3422 len 32768 data: 3 3 6 6 7 7 7 7
27657 223/300 Write block 1344 len 24576 data: 6 6 6 6 6 7
27658 224/300 Write block 1805 len 24576 data: 7 7 7 7 6 6
27659 225/300 Read block 1862 len 4096
27660 226/300 Write block 2118 len 32768 data: 4 4 4 4 3 3 3 3
27661 227/300 Read block 4800 len 20480
27662 228/300 Write block 25 len 24576 data: 7 9 9 9 7 5
27663 229/300 Read block 3187 len 8192
27664 230/300 Read block 4096 len 8192
27665 231/300 Read block 222 len 20480
27666 232/300 Write block 3121 len 4096 data: 3
27667 233/300 Write block 1310 len 20480 data: 5 6 6 6 7
27668 234/300 Write block 1087 len 16384 data: 4 5 7 4
27669 235/300 Read block 3465 len 32768
27670 236/300 Write block 4098 len 36864 data: 3 4 5 5 6 7 7 8 7
27671 237/300 Flush
27672 238/300 Write block 481 len 8192 data: 6 7
27673 239/300 Write block 2661 len 8192 data: 5 7
27674 240/300 Read block 1924 len 12288
27675 241/300 Read block 820 len 20480
27676 242/300 Write block 2300 len 8192 data: 3 5
27677 243/300 Read block 1714 len 16384
27678 244/300 Flush
27679 245/300 Write block 1211 len 32768 data: 6 6 6 6 5 6 6 6
27680 246/300 Write block 4494 len 36864 data: 3 4 4 3 3 4 4 4 5
27681 247/300 Read block 776 len 28672
27682 248/300 Read block 4415 len 40960
27683 249/300 Read block 4081 len 20480
27684 250/300 Read block 3246 len 20480
27685 251/300 Write block 285 len 4096 data: 5
27686 252/300 Read block 4230 len 24576
27687 253/300 Read block 1548 len 12288
27688 254/300 Write block 2559 len 28672 data: 5 6 5 4 5 5 6
27689 255/300 Read block 3862 len 24576
27690 256/300 Write block 4877 len 24576 data: 5 4 3 4 4 5
27691 257/300 Write block 3825 len 24576 data: 5 5 5 8 8 6
27692 258/300 Read block 4993 len 28672
27693 259/300 Write block 1156 len 8192 data: 7 6
27694 260/300 Write block 3495 len 24576 data: 9 5 5 5 4 4
27695 261/300 Write block 4245 len 4096 data: 3
27696 262/300 Read block 289 len 12288
27697 263/300 Read block 1855 len 40960
27698 264/300 Write block 2721 len 28672 data: 7 6 6 7 7 7 6
27699 265/300 Flush
27700 266/300 Write block 1546 len 20480 data: 3 3 5 4 4
27701 267/300 Read block 1200 len 32768
27702 268/300 Read block 2092 len 8192
27703 269/300 Read block 285 len 16384
27704 270/300 Read block 1260 len 20480
27705 271/300 Read block 4170 len 4096
27706 272/300 Write block 342 len 40960 data: 7 6 8 9 7 7 5 4 4 4
27707 273/300 Read block 235 len 28672
27708 274/300 Write block 2631 len 12288 data: 6 5 5
27709 275/300 Write block 3003 len 4096 data: 2
27710 276/300 Read block 4908 len 12288
27711 277/300 Read block 2690 len 24576
27712 278/300 Flush
27713 279/300 Read block 3540 len 8192
27714 280/300 Write block 3079 len 8192 data: 3 6
27715 281/300 Read block 2852 len 28672
27716 282/300 Write block 3226 len 32768 data: 3 4 4 4 5 4 4 4
27717 283/300 Write block 4355 len 20480 data: 4 4 5 4 4
27718 284/300 Write block 202 len 4096 data: 4
27719 285/300 Read block 1818 len 36864
27720 286/300 Write block 1201 len 24576 data: 5 5 3 4 5 6
27721 287/300 Flush
27722 288/300 Write block 1374 len 28672 data: 7 6 7 6 4 4 4
27723 289/300 Write block 4449 len 8192 data: 6 6
27724 290/300 Read block 2023 len 32768
27725 291/300 Write block 716 len 24576 data: 4 3 3 4 3 4
27726 292/300 Write block 1272 len 16384 data: 6 6 5 5
27727 293/300 Read block 4458 len 8192
27728 294/300 Read block 2401 len 4096
27729 295/300 Read block 4133 len 12288
27730 296/300 Write block 4081 len 32768 data: 3 2 2 3 4 4 5 5
27731 297/300 Read block 2544 len 16384
27732 298/300 Write block 505 len 4096 data: 3
27733 299/300 Read block 4766 len 20480
27734 300/300 Write block 2156 len 32768 data: 3 4 6 5 4 4 5 5
27735 301/300 Read block 4835 len 32768
27736 Main task has work to do, go find it
27737 got dsc Start(0)
27738 start 0
27739 Nov 07 14:09:09.197 INFO request completed, latency_us: 207, response_code: 204, uri: /start/cid/0, method: POST, req_id: bd8fe4c2-27c6-4e63-b68d-cb7a388df54d, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
27740 Replay: started 0, returned:Ok(())
27741 ----------------------------------------------------------------
27742 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27743 States: Offline Active Active
27744 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27745 8257 Acked Write 10 Sent Done Done false
27746 8258 Acked Write 5 Sent Done Done false
27747 8259 Acked Read 4 Sent Done Done false
27748 8260 Acked Read 10 Sent Done Done false
27749 8261 Acked Read 2 Sent Done Done false
27750 8262 Acked Read 7 Sent Done Done false
27751 8263 Acked Read 1 Sent Done Done false
27752 8264 Acked Read 7 Sent Done Done false
27753 8265 Acked Write 8 Sent Done Done false
27754 8266 Acked Read 10 Sent Done Done false
27755 8267 Acked Write 9 Sent Done Done false
27756 8268 Acked Read 2 Sent Done Done false
27757 8269 Acked Flush 0 Sent Done Done false
27758 8270 Acked Read 6 Sent Done Done false
27759 8271 Acked Flush 0 Sent Done Done false
27760 8272 Acked Write 1 Sent Done Done false
27761 8273 Acked Read 4 Sent Done Done false
27762 8274 Acked Read 7 Sent Done Done false
27763 8275 Acked Write 10 Sent Done Done false
27764 8276 Acked Flush 0 Sent Done Done false
27765 8277 Acked Read 5 Sent Done Done false
27766 8278 Acked Write 5 Sent Done Done false
27767 8279 Acked Write 10 Sent Done Done false
27768 8280 Acked Write 9 Sent Done Done false
27769 8281 Acked Read 1 Sent Done Done false
27770 8282 Acked Write 3 Sent Done Done false
27771 8283 Acked Read 5 Sent Done Done false
27772 8284 Acked Write 1 Sent Done Done false
27773 8285 Acked Write 3 Sent Done Done false
27774 8286 Acked Read 2 Sent Done Done false
27775 8287 Acked Read 9 Sent Done Done false
27776 8288 Acked Read 9 Sent Done Done false
27777 8289 Acked Write 5 Sent Done Done false
27778 8290 Acked Read 7 Sent Done Done false
27779 8291 Acked Write 1 Sent Done Done false
27780 8292 Acked Read 5 Sent Done Done false
27781 8293 Acked Read 6 Sent Done Done false
27782 8294 Acked Write 10 Sent Done Done false
27783 8295 Acked Read 5 Sent Done Done false
27784 8296 Acked Flush 0 Sent Done Done false
27785 8297 Acked Write 10 Sent Done Done false
27786 8298 Acked Write 8 Sent Done Done false
27787 8299 Acked Read 9 Sent Done Done false
27788 8300 Acked Write 6 Sent Done Done false
27789 8301 Acked Read 3 Sent Done Done false
27790 8302 Acked Read 10 Sent Done Done false
27791 8303 Acked Read 3 Sent Done Done false
27792 8304 Acked Write 8 Sent Done Done false
27793 8305 Acked Flush 0 Sent Done Done false
27794 8306 Acked Read 1 Sent Done Done false
27795 8307 Acked Write 7 Sent Done Done false
27796 8308 Acked Write 5 Sent Done Done false
27797 8309 Acked Write 5 Sent Done Done false
27798 8310 Acked Read 2 Sent Done Done false
27799 8311 Acked Read 3 Sent Done Done false
27800 8312 Acked Write 7 Sent Done Done false
27801 8313 Acked Read 5 Sent Done Done false
27802 8314 Acked Read 3 Sent Done Done false
27803 8315 Acked Write 7 Sent Done Done false
27804 8316 Acked Read 5 Sent Done Done false
27805 8317 Acked Write 5 Sent Done Done false
27806 8318 Acked Read 5 Sent Done Done false
27807 8319 Acked Flush 0 Sent Done Done false
27808 8320 Acked Write 9 Sent Done Done false
27809 8321 Acked Read 7 Sent Done Done false
27810 8322 Acked Read 8 Sent Done Done false
27811 8323 Acked Write 8 Sent Done Done false
27812 8324 Acked Flush 0 Sent Done Done false
27813 8325 Acked Read 4 Sent Done Done false
27814 8326 Acked Flush 0 Sent Done Done false
27815 8327 Acked Write 2 Sent Done Done false
27816 8328 Acked Write 10 Sent Done Done false
27817 8329 Acked Write 4 Sent Done Done false
27818 8330 Acked Read 4 Sent Done Done false
27819 8331 Acked Read 5 Sent Done Done false
27820 8332 Acked Write 4 Sent Done Done false
27821 8333 Acked Write 10 Sent Done Done false
27822 8334 Acked Write 10 Sent Done Done false
27823 8335 Acked Write 9 Sent Done Done false
27824 8336 Acked Read 4 Sent Done Done false
27825 8337 Acked Write 8 Sent Done Done false
27826 8338 Acked Write 6 Sent Done Done false
27827 8339 Acked Read 2 Sent Done Done false
27828 8340 Acked Read 1 Sent Done Done false
27829 8341 Acked Write 5 Sent Done Done false
27830 8342 Acked Read 7 Sent Done Done false
27831 8343 Acked Read 3 Sent Done Done false
27832 8344 Acked Read 3 Sent Done Done false
27833 8345 Acked Read 9 Sent Done Done false
27834 8346 Acked Write 3 Sent Done Done false
27835 8347 Acked Read 6 Sent Done Done false
27836 8348 Acked Read 10 Sent Done Done false
27837 8349 Acked Write 2 Sent Done Done false
27838 8350 Acked Read 1 Sent Done Done false
27839 8351 Acked Read 2 Sent Done Done false
27840 8352 Acked Read 2 Sent Done Done false
27841 8353 Acked Read 5 Sent Done Done false
27842 8354 Acked Write 6 Sent Done Done false
27843 8355 Acked Write 4 Sent Done Done false
27844 8356 Acked Read 7 Sent Done Done false
27845 8357 Acked Read 4 Sent Done Done false
27846 8358 Acked Write 10 Sent Done Done false
27847 8359 Acked Flush 0 Sent Done Done false
27848 8360 Acked Write 3 Sent Done Done false
27849 8361 Acked Write 8 Sent Done Done false
27850 8362 Acked Flush 0 Sent Done Done false
27851 8363 Acked Write 5 Sent Done Done false
27852 8364 Acked Write 10 Sent Done Done false
27853 8365 Acked Read 10 Sent Done Done false
27854 8366 Acked Write 7 Sent Done Done false
27855 8367 Acked Read 9 Sent Done Done false
27856 8368 Acked Write 9 Sent Done Done false
27857 8369 Acked Read 4 Sent Done Done false
27858 8370 Acked Read 3 Sent Done Done false
27859 8371 Acked Read 2 Sent Done Done false
27860 8372 Acked Read 5 Sent Done Done false
27861 8373 Acked Read 6 Sent Done Done false
27862 8374 Acked Read 9 Sent Done Done false
27863 8375 Acked Write 7 Sent Done Done false
27864 8376 Acked Read 3 Sent Done Done false
27865 8377 Acked Read 10 Sent Done Done false
27866 8378 Acked Read 6 Sent Done Done false
27867 8379 Acked Read 2 Sent Done Done false
27868 8380 Acked Write 3 Sent Done Done false
27869 8381 Acked Read 6 Sent Done Done false
27870 8382 Acked Write 7 Sent Done Done false
27871 8383 Acked Read 8 Sent Done Done false
27872 8384 Acked Write 7 Sent Done Done false
27873 8385 Acked Write 10 Sent Done Done false
27874 8386 Acked Read 9 Sent Done Done false
27875 8387 Acked Write 8 Sent Done Done false
27876 8388 Acked Read 7 Sent Done Done false
27877 8389 Acked Read 8 Sent Done Done false
27878 8390 Acked Read 9 Sent Done Done false
27879 8391 Acked Write 10 Sent Done Done false
27880 8392 Acked Read 10 Sent Done Done false
27881 8393 Acked Read 6 Sent Done Done false
27882 8394 Acked Write 3 Sent Done Done false
27883 8395 Acked Write 4 Sent Done Done false
27884 8396 Acked Write 3 Sent Done Done false
27885 8397 Acked Flush 0 Sent Done Done false
27886 8398 Acked Write 5 Sent Done Done false
27887 8399 Acked Read 4 Sent Done Done false
27888 8400 Acked Read 3 Sent Done Done false
27889 8401 Acked Read 3 Sent Done Done false
27890 8402 Acked Write 4 Sent Done Done false
27891 8403 Acked Read 3 Sent Done Done false
27892 8404 Acked Read 8 Sent Done Done false
27893 8405 Acked Write 9 Sent Done Done false
27894 8406 Acked Read 6 Sent Done Done false
27895 8407 Acked Read 9 Sent Done Done false
27896 8408 Acked Read 2 Sent Done Done false
27897 8409 Acked Read 9 Sent Done Done false
27898 8410 Acked Read 8 Sent Done Done false
27899 8411 Acked Write 9 Sent Done Done false
27900 8412 Acked Read 6 Sent Done Done false
27901 8413 Acked Read 3 Sent Done Done false
27902 8414 Acked Read 4 Sent Done Done false
27903 8415 Acked Flush 0 Sent Done Done false
27904 8416 Acked Read 10 Sent Done Done false
27905 8417 Acked Write 6 Sent Done Done false
27906 8418 Acked Read 6 Sent Done Done false
27907 8419 Acked Write 8 Sent Done Done false
27908 8420 Acked Read 3 Sent Done Done false
27909 8421 Acked Read 6 Sent Done Done false
27910 8422 Acked Read 2 Sent Done Done false
27911 8423 Acked Read 4 Sent Done Done false
27912 8424 Acked Write 6 Sent Done Done false
27913 8425 Acked Read 5 Sent Done Done false
27914 8426 Acked Write 8 Sent Done Done false
27915 8427 Acked Write 9 Sent Done Done false
27916 8428 Acked Read 2 Sent Done Done false
27917 8429 Acked Flush 0 Sent Done Done false
27918 8430 Acked Flush 0 Sent Done Done false
27919 8431 Acked Read 4 Sent Done Done false
27920 8432 Acked Write 6 Sent Done Done false
27921 8433 Acked Write 3 Sent Done Done false
27922 8434 Acked Read 9 Sent Done Done false
27923 8435 Acked Read 8 Sent Done Done false
27924 8436 Acked Write 8 Sent Done Done false
27925 8437 Acked Write 7 Sent Done Done false
27926 8438 Acked Write 1 Sent Done Done false
27927 8439 Acked Write 1 Sent Done Done false
27928 8440 Acked Write 5 Sent Done Done false
27929 8441 Acked Read 9 Sent Done Done false
27930 8442 Acked Read 7 Sent Done Done false
27931 8443 Acked Read 5 Sent Done Done false
27932 8444 Acked Write 9 Sent Done Done false
27933 8445 Acked Read 1 Sent Done Done false
27934 8446 Acked Read 5 Sent Done Done false
27935 8447 Acked Write 9 Sent Done Done false
27936 8448 Acked Write 10 Sent Done Done false
27937 8449 Acked Read 3 Sent Done Done false
27938 8450 Acked Read 5 Sent Done Done false
27939 8451 Acked Read 4 Sent Done Done false
27940 8452 Acked Read 8 Sent Done Done false
27941 8453 Acked Read 3 Sent Done Done false
27942 8454 Acked Write 1 Sent Done Done false
27943 8455 Acked Read 8 Sent Done Done false
27944 8456 Acked Read 4 Sent Done Done false
27945 8457 Acked Read 2 Sent Done Done false
27946 8458 Acked Read 9 Sent Done Done false
27947 8459 Acked Read 1 Sent Done Done false
27948 8460 Acked Write 9 Sent Done Done false
27949 8461 Acked Read 8 Sent Done Done false
27950 8462 Acked Write 4 Sent Done Done false
27951 8463 Acked Write 9 Sent Done Done false
27952 8464 Acked Write 5 Sent Done Done false
27953 8465 Acked Read 8 Sent Done Done false
27954 8466 Acked Flush 0 Sent Done Done false
27955 8467 Acked Write 10 Sent Done Done false
27956 8468 Acked Read 7 Sent Done Done false
27957 8469 Acked Read 9 Sent Done Done false
27958 8470 Acked Read 7 Sent Done Done false
27959 8471 Acked Read 4 Sent Done Done false
27960 8472 Acked Read 10 Sent Done Done false
27961 8473 Acked Write 6 Sent Done Done false
27962 8474 Acked Write 2 Sent Done Done false
27963 8475 Acked Flush 0 Sent Done Done false
27964 8476 Acked Read 9 Sent Done Done false
27965 8477 Acked Write 2 Sent Done Done false
27966 8478 Acked Read 6 Sent Done Done false
27967 8479 Acked Write 8 Sent Done Done false
27968 8480 Acked Write 6 Sent Done Done false
27969 8481 Acked Write 6 Sent Done Done false
27970 8482 Acked Read 1 Sent Done Done false
27971 8483 Acked Write 8 Sent Done Done false
27972 8484 Acked Read 5 Sent Done Done false
27973 8485 Acked Write 6 Sent Done Done false
27974 8486 Acked Read 2 Sent Done Done false
27975 8487 Acked Read 2 Sent Done Done false
27976 8488 Acked Read 5 Sent Done Done false
27977 8489 Acked Write 1 Sent Done Done false
27978 8490 Acked Write 5 Sent Done Done false
27979 8491 Acked Write 4 Sent Done Done false
27980 8492 Acked Read 8 Sent Done Done false
27981 8493 Acked Write 9 Sent Done Done false
27982 8494 Acked Flush 0 Sent Done Done false
27983 8495 Acked Write 2 Sent Done Done false
27984 8496 Acked Write 2 Sent Done Done false
27985 8497 Acked Read 3 Sent Done Done false
27986 8498 Acked Read 5 Sent Done Done false
27987 8499 Acked Write 2 Sent Done Done false
27988 8500 Acked Read 4 Sent Done Done false
27989 8501 Acked Flush 0 Sent Done Done false
27990 8502 Acked Write 8 Sent Done Done false
27991 8503 Acked Write 9 Sent Done Done false
27992 8504 Acked Read 7 Sent Done Done false
27993 8505 Acked Read 10 Sent Done Done false
27994 8506 Acked Read 5 Sent Done Done false
27995 8507 Acked Read 5 Sent Done Done false
27996 8508 Acked Write 1 Sent Done Done false
27997 8509 Acked Read 6 Sent Done Done false
27998 8510 Acked Read 3 Sent Done Done false
27999 8511 Acked Write 7 Sent Done Done false
28000 8512 Acked Read 6 Sent Done Done false
28001 8513 Acked Write 6 Sent Done Done false
28002 8514 Acked Write 6 Sent Done Done false
28003 8515 Acked Read 7 Sent Done Done false
28004 8516 Acked Write 2 Sent Done Done false
28005 8517 Acked Write 6 Sent Done Done false
28006 8518 Acked Write 1 Sent Done Done false
28007 8519 Acked Read 3 Sent Done Done false
28008 8520 Acked Read 10 Sent Done Done false
28009 8521 Acked Write 7 Sent Done Done false
28010 8522 Acked Flush 0 Sent Done Done false
28011 8523 Acked Write 5 Sent Done Done false
28012 8524 Acked Read 8 Sent Done Done false
28013 8525 Acked Read 2 Sent Done Done false
28014 8526 Acked Read 4 Sent Done Done false
28015 8527 Acked Read 5 Sent Done Done false
28016 8528 Acked Read 1 Sent Done Done false
28017 8529 Acked Write 10 Sent Done Done false
28018 8530 Acked Read 7 Sent Done Done false
28019 8531 Acked Write 3 Sent Done Done false
28020 8532 Acked Write 1 Sent Done Done false
28021 8533 Acked Read 3 Sent Done Done false
28022 8534 Acked Read 6 Sent Done Done false
28023 8535 Acked Flush 0 Sent Done Done false
28024 8536 Acked Read 2 Sent Done Done false
28025 8537 Acked Write 2 Sent Done Done false
28026 8538 Acked Read 7 Sent Done Done false
28027 8539 Acked Write 8 Sent Done Done false
28028 8540 Acked Write 5 Sent Done Done false
28029 8541 Acked Write 1 Sent Done Done false
28030 8542 Acked Read 9 Sent Done Done false
28031 8543 Acked Write 6 Sent Done Done false
28032 8544 Acked Flush 0 Sent Done Done false
28033 8545 Acked Write 7 Sent Done Done false
28034 8546 Acked Write 2 Sent Done Done false
28035 8547 Acked Read 8 Sent Done Done false
28036 8548 Acked Write 6 Sent Done Done false
28037 8549 Acked Write 4 Sent Done Done false
28038 8550 Acked Read 2 Sent Done Done false
28039 8551 Acked Read 1 Sent Done Done false
28040 8552 Acked Read 3 Sent Done Done false
28041 8553 Acked Write 8 Sent Done Done false
28042 8554 Acked Read 4 Sent Done Done false
28043 8555 Acked Write 1 Sent Done Done false
28044 8556 Acked Read 5 Sent Done Done false
28045 8557 Acked Write 8 Sent Done Done false
28046 8558 Acked Read 8 Sent Done Done false
28047 STATES DS:0 DS:1 DS:2 TOTAL
28048 Sent 302 0 0 302
28049 Done 0 302 302 604
28050 Skipped 0 0 0 0
28051 Error 0 0 0 0
28052 Last Flush: 8256 8544 8544
28053 Downstairs last five completed: 8256 8255 8254 8253 8252
28054 Upstairs last five completed: 8558 8557 8556 8555 8554
28055 CLIENT: Up:0 ds:302 act:2
28056 [8810] got start action
28057 [8810] I am going to restart
28058 Starting downstairs at port 8810
28059 Make output file at "/tmp/dsc/downstairs-8810.txt"
28060 [8810][0] reports Starting
28061 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1224)
28062 [8810][0] reports Running
28063 ----------------------------------------------------------------
28064 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
28065 States: Offline Active Active
28066 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28067 8257 Acked Write 10 Sent Done Done false
28068 8258 Acked Write 5 Sent Done Done false
28069 8259 Acked Read 4 Sent Done Done false
28070 8260 Acked Read 10 Sent Done Done false
28071 8261 Acked Read 2 Sent Done Done false
28072 8262 Acked Read 7 Sent Done Done false
28073 8263 Acked Read 1 Sent Done Done false
28074 8264 Acked Read 7 Sent Done Done false
28075 8265 Acked Write 8 Sent Done Done false
28076 8266 Acked Read 10 Sent Done Done false
28077 8267 Acked Write 9 Sent Done Done false
28078 8268 Acked Read 2 Sent Done Done false
28079 8269 Acked Flush 0 Sent Done Done false
28080 8270 Acked Read 6 Sent Done Done false
28081 8271 Acked Flush 0 Sent Done Done false
28082 8272 Acked Write 1 Sent Done Done false
28083 8273 Acked Read 4 Sent Done Done false
28084 8274 Acked Read 7 Sent Done Done false
28085 8275 Acked Write 10 Sent Done Done false
28086 8276 Acked Flush 0 Sent Done Done false
28087 8277 Acked Read 5 Sent Done Done false
28088 8278 Acked Write 5 Sent Done Done false
28089 8279 Acked Write 10 Sent Done Done false
28090 8280 Acked Write 9 Sent Done Done false
28091 8281 Acked Read 1 Sent Done Done false
28092 8282 Acked Write 3 Sent Done Done false
28093 8283 Acked Read 5 Sent Done Done false
28094 8284 Acked Write 1 Sent Done Done false
28095 8285 Acked Write 3 Sent Done Done false
28096 8286 Acked Read 2 Sent Done Done false
28097 8287 Acked Read 9 Sent Done Done false
28098 8288 Acked Read 9 Sent Done Done false
28099 8289 Acked Write 5 Sent Done Done false
28100 8290 Acked Read 7 Sent Done Done false
28101 8291 Acked Write 1 Sent Done Done false
28102 8292 Acked Read 5 Sent Done Done false
28103 8293 Acked Read 6 Sent Done Done false
28104 8294 Acked Write 10 Sent Done Done false
28105 8295 Acked Read 5 Sent Done Done false
28106 8296 Acked Flush 0 Sent Done Done false
28107 8297 Acked Write 10 Sent Done Done false
28108 8298 Acked Write 8 Sent Done Done false
28109 8299 Acked Read 9 Sent Done Done false
28110 8300 Acked Write 6 Sent Done Done false
28111 8301 Acked Read 3 Sent Done Done false
28112 8302 Acked Read 10 Sent Done Done false
28113 8303 Acked Read 3 Sent Done Done false
28114 8304 Acked Write 8 Sent Done Done false
28115 8305 Acked Flush 0 Sent Done Done false
28116 8306 Acked Read 1 Sent Done Done false
28117 8307 Acked Write 7 Sent Done Done false
28118 8308 Acked Write 5 Sent Done Done false
28119 8309 Acked Write 5 Sent Done Done false
28120 8310 Acked Read 2 Sent Done Done false
28121 8311 Acked Read 3 Sent Done Done false
28122 8312 Acked Write 7 Sent Done Done false
28123 8313 Acked Read 5 Sent Done Done false
28124 8314 Acked Read 3 Sent Done Done false
28125 8315 Acked Write 7 Sent Done Done false
28126 8316 Acked Read 5 Sent Done Done false
28127 8317 Acked Write 5 Sent Done Done false
28128 8318 Acked Read 5 Sent Done Done false
28129 8319 Acked Flush 0 Sent Done Done false
28130 8320 Acked Write 9 Sent Done Done false
28131 8321 Acked Read 7 Sent Done Done false
28132 8322 Acked Read 8 Sent Done Done false
28133 8323 Acked Write 8 Sent Done Done false
28134 8324 Acked Flush 0 Sent Done Done false
28135 8325 Acked Read 4 Sent Done Done false
28136 8326 Acked Flush 0 Sent Done Done false
28137 8327 Acked Write 2 Sent Done Done false
28138 8328 Acked Write 10 Sent Done Done false
28139 8329 Acked Write 4 Sent Done Done false
28140 8330 Acked Read 4 Sent Done Done false
28141 8331 Acked Read 5 Sent Done Done false
28142 8332 Acked Write 4 Sent Done Done false
28143 8333 Acked Write 10 Sent Done Done false
28144 8334 Acked Write 10 Sent Done Done false
28145 8335 Acked Write 9 Sent Done Done false
28146 8336 Acked Read 4 Sent Done Done false
28147 8337 Acked Write 8 Sent Done Done false
28148 8338 Acked Write 6 Sent Done Done false
28149 8339 Acked Read 2 Sent Done Done false
28150 8340 Acked Read 1 Sent Done Done false
28151 8341 Acked Write 5 Sent Done Done false
28152 8342 Acked Read 7 Sent Done Done false
28153 8343 Acked Read 3 Sent Done Done false
28154 8344 Acked Read 3 Sent Done Done false
28155 8345 Acked Read 9 Sent Done Done false
28156 8346 Acked Write 3 Sent Done Done false
28157 8347 Acked Read 6 Sent Done Done false
28158 8348 Acked Read 10 Sent Done Done false
28159 8349 Acked Write 2 Sent Done Done false
28160 8350 Acked Read 1 Sent Done Done false
28161 8351 Acked Read 2 Sent Done Done false
28162 8352 Acked Read 2 Sent Done Done false
28163 8353 Acked Read 5 Sent Done Done false
28164 8354 Acked Write 6 Sent Done Done false
28165 8355 Acked Write 4 Sent Done Done false
28166 8356 Acked Read 7 Sent Done Done false
28167 8357 Acked Read 4 Sent Done Done false
28168 8358 Acked Write 10 Sent Done Done false
28169 8359 Acked Flush 0 Sent Done Done false
28170 8360 Acked Write 3 Sent Done Done false
28171 8361 Acked Write 8 Sent Done Done false
28172 8362 Acked Flush 0 Sent Done Done false
28173 8363 Acked Write 5 Sent Done Done false
28174 8364 Acked Write 10 Sent Done Done false
28175 8365 Acked Read 10 Sent Done Done false
28176 8366 Acked Write 7 Sent Done Done false
28177 8367 Acked Read 9 Sent Done Done false
28178 8368 Acked Write 9 Sent Done Done false
28179 8369 Acked Read 4 Sent Done Done false
28180 8370 Acked Read 3 Sent Done Done false
28181 8371 Acked Read 2 Sent Done Done false
28182 8372 Acked Read 5 Sent Done Done false
28183 8373 Acked Read 6 Sent Done Done false
28184 8374 Acked Read 9 Sent Done Done false
28185 8375 Acked Write 7 Sent Done Done false
28186 8376 Acked Read 3 Sent Done Done false
28187 8377 Acked Read 10 Sent Done Done false
28188 8378 Acked Read 6 Sent Done Done false
28189 8379 Acked Read 2 Sent Done Done false
28190 8380 Acked Write 3 Sent Done Done false
28191 8381 Acked Read 6 Sent Done Done false
28192 8382 Acked Write 7 Sent Done Done false
28193 8383 Acked Read 8 Sent Done Done false
28194 8384 Acked Write 7 Sent Done Done false
28195 8385 Acked Write 10 Sent Done Done false
28196 8386 Acked Read 9 Sent Done Done false
28197 8387 Acked Write 8 Sent Done Done false
28198 8388 Acked Read 7 Sent Done Done false
28199 8389 Acked Read 8 Sent Done Done false
28200 8390 Acked Read 9 Sent Done Done false
28201 8391 Acked Write 10 Sent Done Done false
28202 8392 Acked Read 10 Sent Done Done false
28203 8393 Acked Read 6 Sent Done Done false
28204 8394 Acked Write 3 Sent Done Done false
28205 8395 Acked Write 4 Sent Done Done false
28206 8396 Acked Write 3 Sent Done Done false
28207 8397 Acked Flush 0 Sent Done Done false
28208 8398 Acked Write 5 Sent Done Done false
28209 8399 Acked Read 4 Sent Done Done false
28210 8400 Acked Read 3 Sent Done Done false
28211 8401 Acked Read 3 Sent Done Done false
28212 8402 Acked Write 4 Sent Done Done false
28213 8403 Acked Read 3 Sent Done Done false
28214 8404 Acked Read 8 Sent Done Done false
28215 8405 Acked Write 9 Sent Done Done false
28216 8406 Acked Read 6 Sent Done Done false
28217 8407 Acked Read 9 Sent Done Done false
28218 8408 Acked Read 2 Sent Done Done false
28219 8409 Acked Read 9 Sent Done Done false
28220 8410 Acked Read 8 Sent Done Done false
28221 8411 Acked Write 9 Sent Done Done false
28222 8412 Acked Read 6 Sent Done Done false
28223 8413 Acked Read 3 Sent Done Done false
28224 8414 Acked Read 4 Sent Done Done false
28225 8415 Acked Flush 0 Sent Done Done false
28226 8416 Acked Read 10 Sent Done Done false
28227 8417 Acked Write 6 Sent Done Done false
28228 8418 Acked Read 6 Sent Done Done false
28229 8419 Acked Write 8 Sent Done Done false
28230 8420 Acked Read 3 Sent Done Done false
28231 8421 Acked Read 6 Sent Done Done false
28232 8422 Acked Read 2 Sent Done Done false
28233 8423 Acked Read 4 Sent Done Done false
28234 8424 Acked Write 6 Sent Done Done false
28235 8425 Acked Read 5 Sent Done Done false
28236 8426 Acked Write 8 Sent Done Done false
28237 8427 Acked Write 9 Sent Done Done false
28238 8428 Acked Read 2 Sent Done Done false
28239 8429 Acked Flush 0 Sent Done Done false
28240 8430 Acked Flush 0 Sent Done Done false
28241 8431 Acked Read 4 Sent Done Done false
28242 8432 Acked Write 6 Sent Done Done false
28243 8433 Acked Write 3 Sent Done Done false
28244 8434 Acked Read 9 Sent Done Done false
28245 8435 Acked Read 8 Sent Done Done false
28246 8436 Acked Write 8 Sent Done Done false
28247 8437 Acked Write 7 Sent Done Done false
28248 8438 Acked Write 1 Sent Done Done false
28249 8439 Acked Write 1 Sent Done Done false
28250 8440 Acked Write 5 Sent Done Done false
28251 8441 Acked Read 9 Sent Done Done false
28252 8442 Acked Read 7 Sent Done Done false
28253 8443 Acked Read 5 Sent Done Done false
28254 8444 Acked Write 9 Sent Done Done false
28255 8445 Acked Read 1 Sent Done Done false
28256 8446 Acked Read 5 Sent Done Done false
28257 8447 Acked Write 9 Sent Done Done false
28258 8448 Acked Write 10 Sent Done Done false
28259 8449 Acked Read 3 Sent Done Done false
28260 8450 Acked Read 5 Sent Done Done false
28261 8451 Acked Read 4 Sent Done Done false
28262 8452 Acked Read 8 Sent Done Done false
28263 8453 Acked Read 3 Sent Done Done false
28264 8454 Acked Write 1 Sent Done Done false
28265 8455 Acked Read 8 Sent Done Done false
28266 8456 Acked Read 4 Sent Done Done false
28267 8457 Acked Read 2 Sent Done Done false
28268 8458 Acked Read 9 Sent Done Done false
28269 8459 Acked Read 1 Sent Done Done false
28270 8460 Acked Write 9 Sent Done Done false
28271 8461 Acked Read 8 Sent Done Done false
28272 8462 Acked Write 4 Sent Done Done false
28273 8463 Acked Write 9 Sent Done Done false
28274 8464 Acked Write 5 Sent Done Done false
28275 8465 Acked Read 8 Sent Done Done false
28276 8466 Acked Flush 0 Sent Done Done false
28277 8467 Acked Write 10 Sent Done Done false
28278 8468 Acked Read 7 Sent Done Done false
28279 8469 Acked Read 9 Sent Done Done false
28280 8470 Acked Read 7 Sent Done Done false
28281 8471 Acked Read 4 Sent Done Done false
28282 8472 Acked Read 10 Sent Done Done false
28283 8473 Acked Write 6 Sent Done Done false
28284 8474 Acked Write 2 Sent Done Done false
28285 8475 Acked Flush 0 Sent Done Done false
28286 8476 Acked Read 9 Sent Done Done false
28287 8477 Acked Write 2 Sent Done Done false
28288 8478 Acked Read 6 Sent Done Done false
28289 8479 Acked Write 8 Sent Done Done false
28290 8480 Acked Write 6 Sent Done Done false
28291 8481 Acked Write 6 Sent Done Done false
28292 8482 Acked Read 1 Sent Done Done false
28293 8483 Acked Write 8 Sent Done Done false
28294 8484 Acked Read 5 Sent Done Done false
28295 8485 Acked Write 6 Sent Done Done false
28296 8486 Acked Read 2 Sent Done Done false
28297 8487 Acked Read 2 Sent Done Done false
28298 8488 Acked Read 5 Sent Done Done false
28299 8489 Acked Write 1 Sent Done Done false
28300 8490 Acked Write 5 Sent Done Done false
28301 8491 Acked Write 4 Sent Done Done false
28302 8492 Acked Read 8 Sent Done Done false
28303 8493 Acked Write 9 Sent Done Done false
28304 8494 Acked Flush 0 Sent Done Done false
28305 8495 Acked Write 2 Sent Done Done false
28306 8496 Acked Write 2 Sent Done Done false
28307 8497 Acked Read 3 Sent Done Done false
28308 8498 Acked Read 5 Sent Done Done false
28309 8499 Acked Write 2 Sent Done Done false
28310 8500 Acked Read 4 Sent Done Done false
28311 8501 Acked Flush 0 Sent Done Done false
28312 8502 Acked Write 8 Sent Done Done false
28313 8503 Acked Write 9 Sent Done Done false
28314 8504 Acked Read 7 Sent Done Done false
28315 8505 Acked Read 10 Sent Done Done false
28316 8506 Acked Read 5 Sent Done Done false
28317 8507 Acked Read 5 Sent Done Done false
28318 8508 Acked Write 1 Sent Done Done false
28319 8509 Acked Read 6 Sent Done Done false
28320 8510 Acked Read 3 Sent Done Done false
28321 8511 Acked Write 7 Sent Done Done false
28322 8512 Acked Read 6 Sent Done Done false
28323 8513 Acked Write 6 Sent Done Done false
28324 8514 Acked Write 6 Sent Done Done false
28325 8515 Acked Read 7 Sent Done Done false
28326 8516 Acked Write 2 Sent Done Done false
28327 8517 Acked Write 6 Sent Done Done false
28328 8518 Acked Write 1 Sent Done Done false
28329 8519 Acked Read 3 Sent Done Done false
28330 8520 Acked Read 10 Sent Done Done false
28331 8521 Acked Write 7 Sent Done Done false
28332 8522 Acked Flush 0 Sent Done Done false
28333 8523 Acked Write 5 Sent Done Done false
28334 8524 Acked Read 8 Sent Done Done false
28335 8525 Acked Read 2 Sent Done Done false
28336 8526 Acked Read 4 Sent Done Done false
28337 8527 Acked Read 5 Sent Done Done false
28338 8528 Acked Read 1 Sent Done Done false
28339 8529 Acked Write 10 Sent Done Done false
28340 8530 Acked Read 7 Sent Done Done false
28341 8531 Acked Write 3 Sent Done Done false
28342 8532 Acked Write 1 Sent Done Done false
28343 8533 Acked Read 3 Sent Done Done false
28344 8534 Acked Read 6 Sent Done Done false
28345 8535 Acked Flush 0 Sent Done Done false
28346 8536 Acked Read 2 Sent Done Done false
28347 8537 Acked Write 2 Sent Done Done false
28348 8538 Acked Read 7 Sent Done Done false
28349 8539 Acked Write 8 Sent Done Done false
28350 8540 Acked Write 5 Sent Done Done false
28351 8541 Acked Write 1 Sent Done Done false
28352 8542 Acked Read 9 Sent Done Done false
28353 8543 Acked Write 6 Sent Done Done false
28354 8544 Acked Flush 0 Sent Done Done false
28355 8545 Acked Write 7 Sent Done Done false
28356 8546 Acked Write 2 Sent Done Done false
28357 8547 Acked Read 8 Sent Done Done false
28358 8548 Acked Write 6 Sent Done Done false
28359 8549 Acked Write 4 Sent Done Done false
28360 8550 Acked Read 2 Sent Done Done false
28361 8551 Acked Read 1 Sent Done Done false
28362 8552 Acked Read 3 Sent Done Done false
28363 8553 Acked Write 8 Sent Done Done false
28364 8554 Acked Read 4 Sent Done Done false
28365 8555 Acked Write 1 Sent Done Done false
28366 8556 Acked Read 5 Sent Done Done false
28367 8557 Acked Write 8 Sent Done Done false
28368 8558 Acked Read 8 Sent Done Done false
28369 8559 Acked Flush 0 Sent Done Done false
28370 STATES DS:0 DS:1 DS:2 TOTAL
28371 Sent 303 0 0 303
28372 Done 0 303 303 606
28373 Skipped 0 0 0 0
28374 Error 0 0 0 0
28375 Last Flush: 8256 8559 8559
28376 Downstairs last five completed: 8256 8255 8254 8253 8252
28377 Upstairs last five completed: 8559 8558 8557 8556 8555
28378 CLIENT: Up:0 ds:303 act:2
283792024-11-07T14:09:14.912ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
283802024-11-07T14:09:14.912ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47950) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
283812024-11-07T14:09:14.913ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
283822024-11-07T14:09:14.914ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
283832024-11-07T14:09:14.914ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
283842024-11-07T14:09:14.914ZINFOcrucible: send last flush ID to this DS: 8256 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
283852024-11-07T14:09:14.914ZINFOcrucible: Replied this last flush ID: 8256 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
283862024-11-07T14:09:14.914ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
283872024-11-07T14:09:14.914ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
283882024-11-07T14:09:14.914ZINFOcrucible: [0] client re-new 303 jobs since flush 8256 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
283892024-11-07T14:09:14.914ZINFOcrucible: [0] Replayed 303 jobs since flush: 8256 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
28390 ----------------------------------------------------------------
28391 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28392 Downstairs last five completed: 8559 8558 8557 8556 8555
28393 Upstairs last five completed: 8559 8558 8557 8556 8555
28394 CLIENT: Up:0 ds:0 act:3
28395 Replay: All jobs finished
28396 Main task has work to do, go find it
28397 got dsc Stop(0)
28398 stop 0
28399 Nov 07 14:09:17.202[8810] Got stop action so:false kr:false
28400 INFO request completed, latency_us: 323, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 406ba113-678d-46c0-ab73-d090d900cda3, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
284012024-11-07T14:09:17.203ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
284022024-11-07T14:09:17.203ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
284032024-11-07T14:09:17.203ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
284042024-11-07T14:09:17.203ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
284052024-11-07T14:09:17.203ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
28406 Nov 07 14:09:17.203 INFO request completed, latency_us: 218, response_code: 200, uri: /state/cid/0, method{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:09:17.203385295Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"":"io task","client":"0","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
28407 : GET, req_id: 495ee8a3-158f-4da7-ba49-976739b3dd04, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
28408 [8810] Exited with: ExitStatus(unix_wait_status(9))
28409 [8810][0] reports Exit
28410 Nov 07 14:09:21.205 INFO request completed, latency_us: 223, response_code: 200, uri: /state/cid/0, method: GET, req_id: b711c67c-21c0-428f-936a-5a03906c54b7, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
28411 001/300 Write block 3209 len 12288 data: 4 5 5
28412 002/300 Write block 210 len 40960 data: 5 6 5 4 4 2 2 2 3 3
28413 003/300 Read block 1653 len 8192
28414 004/300 Read block 3566 len 4096
28415 005/300 Write block 4811 len 8192 data: 4 4
28416 006/300 Flush
28417 007/300 Read block 244 len 8192
28418 008/300 Write block 2860 len 8192 data: 5 5
28419 009/300 Read block 284 len 8192
28420 010/300 Write block 4036 len 36864 data: 6 6 5 4 4 6 6 6 7
28421 011/300 Write block 3483 len 32768 data: 7 7 4 7 7 9 5 6
28422 012/300 Read block 913 len 20480
28423 013/300 Read block 3097 len 20480
28424 014/300 Read block 1574 len 12288
28425 015/300 Read block 2080 len 36864
28426 016/300 Write block 2059 len 28672 data: 5 5 5 3 3 4 3
28427 017/300 Write block 3521 len 16384 data: 4 5 2 2
28428 018/300 Write block 4053 len 12288 data: 3 3 3
28429 019/300 Read block 2404 len 28672
28430 020/300 Read block 289 len 24576
28431 021/300 Flush
28432 022/300 Flush
28433 023/300 Write block 1220 len 32768 data: 5 5 6 6 6 5 4 3
28434 024/300 Read block 3926 len 36864
28435 025/300 Read block 3852 len 12288
28436 026/300 Read block 3115 len 12288
28437 027/300 Write block 4464 len 28672 data: 6 6 5 5 5 4 2
28438 028/300 Write block 4583 len 36864 data: 4 3 3 5 5 6 5 6 7
28439 029/300 Write block 2790 len 36864 data: 7 7 7 7 7 5 6 5 5
28440 030/300 Read block 1559 len 24576
28441 031/300 Write block 4784 len 16384 data: 3 4 4 5
28442 032/300 Read block 4417 len 20480
28443 033/300 Write block 1912 len 36864 data: 3 4 6 6 8 6 6 8 7
28444 034/300 Read block 3624 len 4096
28445 035/300 Flush
28446 036/300 Write block 1725 len 4096 data: 4
28447 037/300 Write block 2652 len 36864 data: 4 3 3 3 4 4 6 6 5
28448 038/300 Flush
28449 039/300 Flush
28450 040/300 Read block 4490 len 28672
28451 041/300 Write block 2566 len 24576 data: 6 6 4 4 3 4
28452 042/300 Flush
28453 043/300 Read block 4107 len 32768
28454 044/300 Write block 1644 len 32768 data: 6 5 4 7 9 8 7 7
28455 045/300 Write block 455 len 20480 data: 7 9 9 6 5
28456 046/300 Write block 3068 len 4096 data: 4
28457 047/300 Write block 3493 len 20480 data: 8 8 10 6 6
28458 048/300 Read block 3911 len 8192
28459 049/300 Read block 1681 len 12288
28460 050/300 Read block 2066 len 12288
28461 051/300 Read block 957 len 20480
28462 052/300 Write block 383 len 20480 data: 7 5 6 6 6
28463 053/300 Flush
28464 054/300 Flush
28465 055/300 Write block 297 len 24576 data: 4 4 4 4 7 5
28466 056/300 Write block 1535 len 12288 data: 3 3 2
28467 057/300 Write block 3947 len 4096 data: 3
28468 058/300 Read block 103 len 40960
28469 059/300 Read block 2115 len 8192
28470 060/300 Write block 2784 len 20480 data: 6 6 7 7 6
28471 061/300 Write block 3752 len 20480 data: 8 9 8 6 5
28472 062/300 Write block 3525 len 8192 data: 3 3
28473 063/300 Write block 3134 len 28672 data: 7 8 8 5 5 4 5
28474 064/300 Read block 275 len 4096
28475 065/300 Write block 2597 len 12288 data: 7 5 6
28476 066/300 Write block 2602 len 28672 data: 4 5 5 4 4 4 4
28477 067/300 Write block 3498 len 28672 data: 6 5 5 5 6 7 7
28478 068/300 Write block 1753 len 32768 data: 8 8 7 6 6 5 5 4
28479 069/300 Write block 4051 len 12288 data: 4 3 4
28480 070/300 Read block 3455 len 4096
28481 071/300 Write block 676 len 24576 data: 4 4 6 6 6 5
28482 072/300 Write block 442 len 40960 data: 8 7 8 7 5 4 4 4 6 6
28483 073/300 Write block 1507 len 32768 data: 8 7 9 6 6 6 5 4
28484 074/300 Write block 695 len 32768 data: 5 5 6 4 4 3 3 3
28485 075/300 Write block 4348 len 16384 data: 7 7 6 6
28486 076/300 Flush
28487 077/300 Write block 2557 len 28672 data: 6 5 6 7 6 5 6
28488 078/300 Flush
28489 079/300 Flush
28490 080/300 Read block 1146 len 8192
28491 081/300 Read block 2367 len 32768
28492 082/300 Read block 2754 len 16384
28493 083/300 Write block 1536 len 36864 data: 4 3 3 4 4 4 4 4 4
28494 084/300 Read block 3518 len 4096
28495 085/300 Write block 4471 len 16384 data: 2 2 3 3
28496 086/300 Read block 4512 len 16384
28497 087/300 Write block 4795 len 12288 data: 8 8 8
28498 088/300 Read block 3940 len 4096
28499 089/300 Read block 2673 len 24576
28500 090/300 Write block 2364 len 16384 data: 6 8 6 5
28501 091/300 Write block 1077 len 8192 data: 5 6
28502 092/300 Read block 2982 len 40960
28503 093/300 Read block 4992 len 12288
28504 094/300 Read block 1573 len 4096
28505 095/300 Read block 955 len 36864
28506 096/300 Write block 4529 len 8192 data: 10 11
28507 097/300 Read block 2246 len 16384
28508 098/300 Write block 1356 len 16384 data: 11 10 10 11
28509 099/300 Write block 4730 len 24576 data: 9 9 7 8 6 5
28510 100/300 Write block 4835 len 36864 data: 2 4 5 5 4 4 8 10 9
28511 101/300 Read block 3366 len 12288
28512 102/300 Write block 2734 len 32768 data: 7 5 5 6 5 3 2 3
28513 103/300 Write block 268 len 28672 data: 5 5 5 3 3 3 3
28514 104/300 Read block 609 len 28672
28515 105/300 Write block 4193 len 40960 data: 4 4 3 3 3 5 5 6 6 6
28516 106/300 Write block 3084 len 12288 data: 10 10 8
28517 107/300 Write block 2333 len 28672 data: 5 4 5 4 4 5 6
28518 108/300 Write block 2112 len 16384 data: 5 5 5 5
28519 109/300 Write block 1396 len 20480 data: 6 7 7 8 7
28520 110/300 Flush
28521 111/300 Read block 1755 len 28672
28522 112/300 Write block 560 len 32768 data: 6 6 6 8 9 6 6 6
28523 113/300 Read block 4605 len 40960
28524 114/300 Write block 3926 len 12288 data: 8 9 7
28525 115/300 Read block 4784 len 32768
28526 116/300 Read block 3050 len 4096
28527 117/300 Read block 2996 len 24576
28528 118/300 Write block 1969 len 32768 data: 6 6 5 6 6 6 4 4
28529 119/300 Flush
28530 120/300 Write block 1939 len 24576 data: 4 6 6 8 9 8
28531 121/300 Read block 2422 len 8192
28532 122/300 Read block 1772 len 28672
28533 123/300 Read block 2676 len 20480
28534 124/300 Write block 605 len 20480 data: 6 6 5 4 5
28535 125/300 Read block 3870 len 40960
28536 126/300 Flush
28537 127/300 Read block 3396 len 28672
28538 128/300 Write block 959 len 24576 data: 4 2 2 3 3 3
28539 129/300 Flush
28540 130/300 Flush
28541 131/300 Flush
28542 132/300 Flush
28543 133/300 Read block 2859 len 28672
28544 134/300 Write block 819 len 8192 data: 4 3
28545 135/300 Write block 3196 len 28672 data: 6 3 5 4 5 5 5
28546 136/300 Read block 469 len 4096
28547 137/300 Write block 996 len 40960 data: 5 4 5 5 6 5 4 3 3 4
28548 138/300 Read block 3545 len 32768
28549 139/300 Read block 1911 len 40960
28550 140/300 Write block 766 len 4096 data: 4
28551 141/300 Read block 277 len 24576
28552 142/300 Write block 604 len 32768 data: 6 7 7 6 5 6 6 6
28553 143/300 Read block 3582 len 24576
28554 144/300 Write block 847 len 36864 data: 4 5 5 5 5 5 4 6 7
28555 145/300 Read block 180 len 32768
28556 146/300 Write block 2179 len 12288 data: 4 4 7
28557 147/300 Read block 2059 len 28672
28558 148/300 Write block 2871 len 8192 data: 2 2
28559 149/300 Flush
28560 150/300 Write block 28 len 24576 data: 10 8 6 5 5 6
28561 151/300 Read block 4359 len 36864
28562 152/300 Read block 4986 len 40960
28563 153/300 Flush
28564 154/300 Read block 1001 len 20480
28565 155/300 Read block 3037 len 8192
28566 156/300 Write block 702 len 36864 data: 4 4 6 6 9 11 10 11 9
28567 157/300 Read block 4620 len 8192
28568 158/300 Read block 3030 len 16384
28569 159/300 Read block 766 len 12288
28570 160/300 Write block 4035 len 24576 data: 6 7 7 6 5 5
28571 161/300 Read block 2228 len 36864
28572 162/300 Write block 4436 len 32768 data: 7 8 9 10 8 9 8 8
28573 163/300 Write block 2822 len 16384 data: 5 6 7 6
28574 164/300 Read block 4775 len 28672
28575 165/300 Read block 1293 len 20480
28576 166/300 Flush
28577 167/300 Write block 1714 len 12288 data: 7 6 5
28578 168/300 Read block 4658 len 8192
28579 169/300 Read block 608 len 20480
28580 170/300 Read block 429 len 40960
28581 171/300 Read block 881 len 40960
28582 172/300 Read block 468 len 40960
28583 173/300 Flush
28584 174/300 Write block 972 len 40960 data: 5 5 4 5 6 6 5 4 7 8
28585 175/300 Read block 3767 len 28672
28586 176/300 Read block 1082 len 36864
28587 177/300 Write block 846 len 28672 data: 3 5 6 6 6 6 6
28588 178/300 Write block 4481 len 28672 data: 8 9 8 7 8 7 7
28589 179/300 Read block 4949 len 24576
28590 180/300 Read block 438 len 24576
28591 181/300 Read block 3978 len 8192
28592 182/300 Read block 2470 len 4096
28593 183/300 Write block 1396 len 12288 data: 7 8 8
28594 184/300 Write block 4987 len 16384 data: 5 6 7 7
28595 185/300 Read block 4115 len 40960
28596 186/300 Write block 3 len 8192 data: 9 8
28597 187/300 Read block 4008 len 40960
28598 188/300 Write block 1198 len 16384 data: 4 5 5 6
28599 189/300 Read block 472 len 36864
28600 190/300 Read block 1251 len 28672
28601 191/300 Write block 4900 len 4096 data: 6
28602 192/300 Read block 2656 len 28672
28603 193/300 Read block 3687 len 40960
28604 194/300 Write block 4000 len 16384 data: 6 6 6 6
28605 195/300 Write block 3092 len 12288 data: 7 7 4
28606 196/300 Read block 3868 len 4096
28607 197/300 Write block 1604 len 8192 data: 8 7
28608 198/300 Read block 4208 len 12288
28609 199/300 Read block 1574 len 12288
28610 200/300 Write block 1607 len 36864 data: 5 4 3 2 4 3 5 5 5
28611 201/300 Flush
28612 202/300 Read block 1346 len 16384
28613 203/300 Flush
28614 204/300 Write block 2643 len 16384 data: 8 6 5 6
28615 205/300 Write block 182 len 12288 data: 7 5 6
28616 206/300 Read block 1095 len 20480
28617 207/300 Write block 3423 len 12288 data: 4 7 7
28618 208/300 Write block 3463 len 12288 data: 6 5 5
28619 209/300 Write block 362 len 20480 data: 5 4 3 3 3
28620 210/300 Write block 3850 len 32768 data: 4 4 3 3 3 3 3 2
28621 211/300 Write block 1745 len 8192 data: 4 4
28622 212/300 Write block 1142 len 24576 data: 7 7 8 7 7 5
28623 213/300 Read block 753 len 36864
28624 214/300 Read block 979 len 12288
28625 215/300 Flush
28626 216/300 Write block 2099 len 16384 data: 3 2 4 4
28627 217/300 Read block 4201 len 24576
28628 218/300 Read block 912 len 16384
28629 219/300 Read block 3931 len 32768
28630 220/300 Write block 1377 len 20480 data: 7 5 5 5 4
28631 221/300 Write block 380 len 4096 data: 9
28632 222/300 Read block 4348 len 40960
28633 223/300 Write block 4764 len 32768 data: 6 7 6 5 4 4 5 6
28634 224/300 Read block 1773 len 4096
28635 225/300 Write block 462 len 16384 data: 4 4 4 5
28636 226/300 Write block 3256 len 36864 data: 3 4 4 4 4 4 5 4 5
28637 227/300 Read block 356 len 16384
28638 228/300 Write block 2101 len 8192 data: 5 5
28639 229/300 Write block 4326 len 40960 data: 2 2 4 3 4 5 5 5 5 3
28640 230/300 Read block 1853 len 12288
28641 231/300 Flush
28642 232/300 Flush
28643 233/300 Read block 4691 len 16384
28644 234/300 Write block 4648 len 12288 data: 9 7 5
28645 235/300 Read block 2875 len 8192
28646 236/300 Read block 3385 len 36864
28647 237/300 Flush
28648 238/300 Write block 69 len 40960 data: 3 4 4 8 7 6 5 5 5 5
28649 239/300 Read block 1692 len 32768
28650 240/300 Write block 1304 len 36864 data: 2 2 3 3 3 5 6 7 7
28651 241/300 Flush
28652 242/300 Read block 1524 len 40960
28653 243/300 Read block 3625 len 32768
28654 244/300 Read block 3273 len 4096
28655 245/300 Read block 2859 len 8192
28656 246/300 Write block 3305 len 28672 data: 5 5 5 6 6 6 6
28657 247/300 Write block 2238 len 28672 data: 4 5 6 8 6 6 6
28658 248/300 Write block 295 len 24576 data: 5 3 5 5 5 5
28659 249/300 Write block 4399 len 8192 data: 4 3
28660 250/300 Read block 950 len 16384
28661 251/300 Write block 2881 len 36864 data: 5 6 7 6 6 5 6 6 4
28662 252/300 Read block 3715 len 32768
28663 253/300 Write block 693 len 8192 data: 4 4
28664 254/300 Write block 4953 len 16384 data: 9 10 11 9
28665 255/300 Write block 1400 len 28672 data: 8 7 6 7 7 7 7
28666 256/300 Flush
28667 257/300 Read block 3581 len 32768
28668 258/300 Flush
28669 259/300 Read block 4059 len 36864
28670 260/300 Write block 3525 len 4096 data: 4
28671 261/300 Flush
28672 262/300 Read block 3085 len 32768
28673 263/300 Write block 1764 len 40960 data: 6 7 5 6 6 8 6 7 6 7
28674 264/300 Write block 830 len 36864 data: 7 6 6 6 6 5 5 2 3
28675 265/300 Read block 927 len 4096
28676 266/300 Write block 4873 len 16384 data: 5 4 5 5
28677 267/300 Read block 2998 len 28672
28678 268/300 Read block 4316 len 40960
28679 269/300 Read block 1095 len 24576
28680 270/300 Write block 116 len 28672 data: 3 3 3 3 3 3 4
28681 271/300 Read block 3404 len 36864
28682 272/300 Write block 4897 len 20480 data: 7 6 6 7 3
28683 273/300 Write block 3910 len 8192 data: 3 4
28684 274/300 Write block 4214 len 12288 data: 5 5 5
28685 275/300 Flush
28686 276/300 Read block 568 len 28672
28687 277/300 Flush
28688 278/300 Read block 3959 len 32768
28689 279/300 Write block 3334 len 36864 data: 8 8 7 6 7 8 7 7 8
28690 280/300 Write block 3013 len 24576 data: 4 5 5 5 7 5
28691 281/300 Write block 2849 len 40960 data: 7 7 7 8 9 10 7 6 6 6
28692 282/300 Write block 3511 len 36864 data: 7 7 6 6 4 4 3 3 3
28693 283/300 Read block 1547 len 24576
28694 284/300 Write block 3034 len 12288 data: 5 5 3
28695 285/300 Read block 3280 len 40960
28696 286/300 Write block 113 len 12288 data: 2 3 3
28697 287/300 Write block 301 len 16384 data: 8 6 6 7
28698 288/300 Read block 1881 len 16384
28699 289/300 Read block 1817 len 20480
28700 290/300 Read block 696 len 16384
28701 291/300 Read block 3200 len 16384
28702 292/300 Read block 2360 len 12288
28703 293/300 Write block 1183 len 32768 data: 4 4 4 5 6 8 8 6
28704 294/300 Write block 4148 len 20480 data: 8 8 9 8 7
28705 295/300 Write block 1222 len 16384 data: 7 7 7 6
28706 296/300 Write block 3137 len 24576 data: 6 6 5 6 5 6
28707 297/300 Write block 3973 len 24576 data: 8 6 5 5 6 5
28708 298/300 Write block 1708 len 16384 data: 4 4 5 8
28709 299/300 Read block 4581 len 40960
28710 300/300 Write block 1829 len 32768 data: 10 11 8 8 6 5 5 5
28711 301/300 Read block 1982 len 20480
28712 Main task has work to do, go find it
28713 got dsc Start(0)
28714 start 0
28715 Nov 07 14:09:21.491 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/0, method: POST, req_id: 6c247988-a229-4d6d-b114-f1bedd568252, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
28716 Replay: started 0, returned:Ok(())
28717 ----------------------------------------------------------------
28718 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28719 States: Offline Active Active
28720 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28721 8560 Acked Write 3 Sent Done Done false
28722 8561 Acked Write 10 Sent Done Done false
28723 8562 Acked Read 2 Sent Done Done false
28724 8563 Acked Read 1 Sent Done Done false
28725 8564 Acked Write 2 Sent Done Done false
28726 8565 Acked Flush 0 Sent Done Done false
28727 8566 Acked Read 2 Sent Done Done false
28728 8567 Acked Write 2 Sent Done Done false
28729 8568 Acked Read 2 Sent Done Done false
28730 8569 Acked Write 9 Sent Done Done false
28731 8570 Acked Write 8 Sent Done Done false
28732 8571 Acked Read 5 Sent Done Done false
28733 8572 Acked Read 5 Sent Done Done false
28734 8573 Acked Read 3 Sent Done Done false
28735 8574 Acked Read 9 Sent Done Done false
28736 8575 Acked Write 7 Sent Done Done false
28737 8576 Acked Write 4 Sent Done Done false
28738 8577 Acked Write 3 Sent Done Done false
28739 8578 Acked Read 7 Sent Done Done false
28740 8579 Acked Read 6 Sent Done Done false
28741 8580 Acked Flush 0 Sent Done Done false
28742 8581 Acked Flush 0 Sent Done Done false
28743 8582 Acked Write 8 Sent Done Done false
28744 8583 Acked Read 9 Sent Done Done false
28745 8584 Acked Read 3 Sent Done Done false
28746 8585 Acked Read 3 Sent Done Done false
28747 8586 Acked Write 7 Sent Done Done false
28748 8587 Acked Write 9 Sent Done Done false
28749 8588 Acked Write 9 Sent Done Done false
28750 8589 Acked Read 6 Sent Done Done false
28751 8590 Acked Write 4 Sent Done Done false
28752 8591 Acked Read 5 Sent Done Done false
28753 8592 Acked Write 9 Sent Done Done false
28754 8593 Acked Read 1 Sent Done Done false
28755 8594 Acked Flush 0 Sent Done Done false
28756 8595 Acked Write 1 Sent Done Done false
28757 8596 Acked Write 9 Sent Done Done false
28758 8597 Acked Flush 0 Sent Done Done false
28759 8598 Acked Flush 0 Sent Done Done false
28760 8599 Acked Read 7 Sent Done Done false
28761 8600 Acked Write 6 Sent Done Done false
28762 8601 Acked Flush 0 Sent Done Done false
28763 8602 Acked Read 8 Sent Done Done false
28764 8603 Acked Write 8 Sent Done Done false
28765 8604 Acked Write 5 Sent Done Done false
28766 8605 Acked Write 1 Sent Done Done false
28767 8606 Acked Write 5 Sent Done Done false
28768 8607 Acked Read 2 Sent Done Done false
28769 8608 Acked Read 3 Sent Done Done false
28770 8609 Acked Read 3 Sent Done Done false
28771 8610 Acked Read 5 Sent Done Done false
28772 8611 Acked Write 5 Sent Done Done false
28773 8612 Acked Flush 0 Sent Done Done false
28774 8613 Acked Flush 0 Sent Done Done false
28775 8614 Acked Write 6 Sent Done Done false
28776 8615 Acked Write 3 Sent Done Done false
28777 8616 Acked Write 1 Sent Done Done false
28778 8617 Acked Read 10 Sent Done Done false
28779 8618 Acked Read 2 Sent Done Done false
28780 8619 Acked Write 5 Sent Done Done false
28781 8620 Acked Write 5 Sent Done Done false
28782 8621 Acked Write 2 Sent Done Done false
28783 8622 Acked Write 7 Sent Done Done false
28784 8623 Acked Read 1 Sent Done Done false
28785 8624 Acked Write 3 Sent Done Done false
28786 8625 Acked Write 7 Sent Done Done false
28787 8626 Acked Write 7 Sent Done Done false
28788 8627 Acked Write 8 Sent Done Done false
28789 8628 Acked Write 3 Sent Done Done false
28790 8629 Acked Read 1 Sent Done Done false
28791 8630 Acked Write 6 Sent Done Done false
28792 8631 Acked Write 10 Sent Done Done false
28793 8632 Acked Write 8 Sent Done Done false
28794 8633 Acked Write 8 Sent Done Done false
28795 8634 Acked Write 4 Sent Done Done false
28796 8635 Acked Flush 0 Sent Done Done false
28797 8636 Acked Write 7 Sent Done Done false
28798 8637 Acked Flush 0 Sent Done Done false
28799 8638 Acked Flush 0 Sent Done Done false
28800 8639 Acked Read 2 Sent Done Done false
28801 8640 Acked Read 8 Sent Done Done false
28802 8641 Acked Read 4 Sent Done Done false
28803 8642 Acked Write 9 Sent Done Done false
28804 8643 Acked Read 1 Sent Done Done false
28805 8644 Acked Write 4 Sent Done Done false
28806 8645 Acked Read 4 Sent Done Done false
28807 8646 Acked Write 3 Sent Done Done false
28808 8647 Acked Read 1 Sent Done Done false
28809 8648 Acked Read 6 Sent Done Done false
28810 8649 Acked Write 4 Sent Done Done false
28811 8650 Acked Write 2 Sent Done Done false
28812 8651 Acked Read 10 Sent Done Done false
28813 8652 Acked Read 3 Sent Done Done false
28814 8653 Acked Read 1 Sent Done Done false
28815 8654 Acked Read 9 Sent Done Done false
28816 8655 Acked Write 2 Sent Done Done false
28817 8656 Acked Read 4 Sent Done Done false
28818 8657 Acked Write 4 Sent Done Done false
28819 8658 Acked Write 6 Sent Done Done false
28820 8659 Acked Write 9 Sent Done Done false
28821 8660 Acked Read 3 Sent Done Done false
28822 8661 Acked Write 8 Sent Done Done false
28823 8662 Acked Write 7 Sent Done Done false
28824 8663 Acked Read 7 Sent Done Done false
28825 8664 Acked Write 10 Sent Done Done false
28826 8665 Acked Write 3 Sent Done Done false
28827 8666 Acked Write 7 Sent Done Done false
28828 8667 Acked Write 4 Sent Done Done false
28829 8668 Acked Write 5 Sent Done Done false
28830 8669 Acked Flush 0 Sent Done Done false
28831 8670 Acked Read 7 Sent Done Done false
28832 8671 Acked Write 8 Sent Done Done false
28833 8672 Acked Read 10 Sent Done Done false
28834 8673 Acked Write 3 Sent Done Done false
28835 8674 Acked Read 8 Sent Done Done false
28836 8675 Acked Read 1 Sent Done Done false
28837 8676 Acked Read 6 Sent Done Done false
28838 8677 Acked Write 8 Sent Done Done false
28839 8678 Acked Flush 0 Sent Done Done false
28840 8679 Acked Write 6 Sent Done Done false
28841 8680 Acked Read 2 Sent Done Done false
28842 8681 Acked Read 7 Sent Done Done false
28843 8682 Acked Read 5 Sent Done Done false
28844 8683 Acked Write 5 Sent Done Done false
28845 8684 Acked Read 10 Sent Done Done false
28846 8685 Acked Flush 0 Sent Done Done false
28847 8686 Acked Read 7 Sent Done Done false
28848 8687 Acked Write 6 Sent Done Done false
28849 8688 Acked Flush 0 Sent Done Done false
28850 8689 Acked Flush 0 Sent Done Done false
28851 8690 Acked Flush 0 Sent Done Done false
28852 8691 Acked Flush 0 Sent Done Done false
28853 8692 Acked Read 7 Sent Done Done false
28854 8693 Acked Write 2 Sent Done Done false
28855 8694 Acked Write 7 Sent Done Done false
28856 8695 Acked Read 1 Sent Done Done false
28857 8696 Acked Write 10 Sent Done Done false
28858 8697 Acked Read 8 Sent Done Done false
28859 8698 Acked Read 10 Sent Done Done false
28860 8699 Acked Write 1 Sent Done Done false
28861 8700 Acked Read 6 Sent Done Done false
28862 8701 Acked Write 8 Sent Done Done false
28863 8702 Acked Read 6 Sent Done Done false
28864 8703 Acked Write 9 Sent Done Done false
28865 8704 Acked Read 8 Sent Done Done false
28866 8705 Acked Write 3 Sent Done Done false
28867 8706 Acked Read 7 Sent Done Done false
28868 8707 Acked Write 2 Sent Done Done false
28869 8708 Acked Flush 0 Sent Done Done false
28870 8709 Acked Write 6 Sent Done Done false
28871 8710 Acked Read 9 Sent Done Done false
28872 8711 Acked Read 10 Sent Done Done false
28873 8712 Acked Flush 0 Sent Done Done false
28874 8713 Acked Read 5 Sent Done Done false
28875 8714 Acked Read 2 Sent Done Done false
28876 8715 Acked Write 9 Sent Done Done false
28877 8716 Acked Read 2 Sent Done Done false
28878 8717 Acked Read 4 Sent Done Done false
28879 8718 Acked Read 3 Sent Done Done false
28880 8719 Acked Write 6 Sent Done Done false
28881 8720 Acked Read 9 Sent Done Done false
28882 8721 Acked Write 8 Sent Done Done false
28883 8722 Acked Write 4 Sent Done Done false
28884 8723 Acked Read 7 Sent Done Done false
28885 8724 Acked Read 5 Sent Done Done false
28886 8725 Acked Flush 0 Sent Done Done false
28887 8726 Acked Write 3 Sent Done Done false
28888 8727 Acked Read 2 Sent Done Done false
28889 8728 Acked Read 5 Sent Done Done false
28890 8729 Acked Read 10 Sent Done Done false
28891 8730 Acked Read 10 Sent Done Done false
28892 8731 Acked Read 10 Sent Done Done false
28893 8732 Acked Flush 0 Sent Done Done false
28894 8733 Acked Write 10 Sent Done Done false
28895 8734 Acked Read 7 Sent Done Done false
28896 8735 Acked Read 9 Sent Done Done false
28897 8736 Acked Write 7 Sent Done Done false
28898 8737 Acked Write 7 Sent Done Done false
28899 8738 Acked Read 6 Sent Done Done false
28900 8739 Acked Read 6 Sent Done Done false
28901 8740 Acked Read 2 Sent Done Done false
28902 8741 Acked Read 1 Sent Done Done false
28903 8742 Acked Write 3 Sent Done Done false
28904 8743 Acked Write 4 Sent Done Done false
28905 8744 Acked Read 10 Sent Done Done false
28906 8745 Acked Write 2 Sent Done Done false
28907 8746 Acked Read 10 Sent Done Done false
28908 8747 Acked Write 4 Sent Done Done false
28909 8748 Acked Read 9 Sent Done Done false
28910 8749 Acked Read 7 Sent Done Done false
28911 8750 Acked Write 1 Sent Done Done false
28912 8751 Acked Read 7 Sent Done Done false
28913 8752 Acked Read 10 Sent Done Done false
28914 8753 Acked Write 4 Sent Done Done false
28915 8754 Acked Write 3 Sent Done Done false
28916 8755 Acked Read 1 Sent Done Done false
28917 8756 Acked Write 2 Sent Done Done false
28918 8757 Acked Read 3 Sent Done Done false
28919 8758 Acked Read 3 Sent Done Done false
28920 8759 Acked Write 9 Sent Done Done false
28921 8760 Acked Flush 0 Sent Done Done false
28922 8761 Acked Read 4 Sent Done Done false
28923 8762 Acked Flush 0 Sent Done Done false
28924 8763 Acked Write 4 Sent Done Done false
28925 8764 Acked Write 3 Sent Done Done false
28926 8765 Acked Read 5 Sent Done Done false
28927 8766 Acked Write 3 Sent Done Done false
28928 8767 Acked Write 3 Sent Done Done false
28929 8768 Acked Write 5 Sent Done Done false
28930 8769 Acked Write 8 Sent Done Done false
28931 8770 Acked Write 2 Sent Done Done false
28932 8771 Acked Write 6 Sent Done Done false
28933 8772 Acked Read 9 Sent Done Done false
28934 8773 Acked Read 3 Sent Done Done false
28935 8774 Acked Flush 0 Sent Done Done false
28936 8775 Acked Write 4 Sent Done Done false
28937 8776 Acked Read 6 Sent Done Done false
28938 8777 Acked Read 4 Sent Done Done false
28939 8778 Acked Read 8 Sent Done Done false
28940 8779 Acked Write 5 Sent Done Done false
28941 8780 Acked Write 1 Sent Done Done false
28942 8781 Acked Read 10 Sent Done Done false
28943 8782 Acked Write 8 Sent Done Done false
28944 8783 Acked Read 1 Sent Done Done false
28945 8784 Acked Write 4 Sent Done Done false
28946 8785 Acked Write 9 Sent Done Done false
28947 8786 Acked Read 4 Sent Done Done false
28948 8787 Acked Write 2 Sent Done Done false
28949 8788 Acked Write 10 Sent Done Done false
28950 8789 Acked Read 3 Sent Done Done false
28951 8790 Acked Flush 0 Sent Done Done false
28952 8791 Acked Flush 0 Sent Done Done false
28953 8792 Acked Read 4 Sent Done Done false
28954 8793 Acked Write 3 Sent Done Done false
28955 8794 Acked Read 2 Sent Done Done false
28956 8795 Acked Read 9 Sent Done Done false
28957 8796 Acked Flush 0 Sent Done Done false
28958 8797 Acked Write 10 Sent Done Done false
28959 8798 Acked Read 8 Sent Done Done false
28960 8799 Acked Write 9 Sent Done Done false
28961 8800 Acked Flush 0 Sent Done Done false
28962 8801 Acked Read 10 Sent Done Done false
28963 8802 Acked Read 8 Sent Done Done false
28964 8803 Acked Read 1 Sent Done Done false
28965 8804 Acked Read 2 Sent Done Done false
28966 8805 Acked Write 7 Sent Done Done false
28967 8806 Acked Write 7 Sent Done Done false
28968 8807 Acked Write 6 Sent Done Done false
28969 8808 Acked Write 2 Sent Done Done false
28970 8809 Acked Read 4 Sent Done Done false
28971 8810 Acked Write 9 Sent Done Done false
28972 8811 Acked Read 8 Sent Done Done false
28973 8812 Acked Write 2 Sent Done Done false
28974 8813 Acked Write 4 Sent Done Done false
28975 8814 Acked Write 7 Sent Done Done false
28976 8815 Acked Flush 0 Sent Done Done false
28977 8816 Acked Read 8 Sent Done Done false
28978 8817 Acked Flush 0 Sent Done Done false
28979 8818 Acked Read 9 Sent Done Done false
28980 8819 Acked Write 1 Sent Done Done false
28981 8820 Acked Flush 0 Sent Done Done false
28982 8821 Acked Read 8 Sent Done Done false
28983 8822 Acked Write 10 Sent Done Done false
28984 8823 Acked Write 9 Sent Done Done false
28985 8824 Acked Read 1 Sent Done Done false
28986 8825 Acked Write 4 Sent Done Done false
28987 8826 Acked Read 7 Sent Done Done false
28988 8827 Acked Read 10 Sent Done Done false
28989 8828 Acked Read 6 Sent Done Done false
28990 8829 Acked Write 7 Sent Done Done false
28991 8830 Acked Read 9 Sent Done Done false
28992 8831 Acked Write 5 Sent Done Done false
28993 8832 Acked Write 2 Sent Done Done false
28994 8833 Acked Write 3 Sent Done Done false
28995 8834 Acked Flush 0 Sent Done Done false
28996 8835 Acked Read 7 Sent Done Done false
28997 8836 Acked Flush 0 Sent Done Done false
28998 8837 Acked Read 8 Sent Done Done false
28999 8838 Acked Write 9 Sent Done Done false
29000 8839 Acked Write 6 Sent Done Done false
29001 8840 Acked Write 10 Sent Done Done false
29002 8841 Acked Write 9 Sent Done Done false
29003 8842 Acked Read 6 Sent Done Done false
29004 8843 Acked Write 3 Sent Done Done false
29005 8844 Acked Read 10 Sent Done Done false
29006 8845 Acked Flush 0 Sent Done Done false
29007 8846 Acked Write 3 Sent Done Done false
29008 8847 Acked Write 4 Sent Done Done false
29009 8848 Acked Read 4 Sent Done Done false
29010 8849 Acked Read 5 Sent Done Done false
29011 8850 Acked Read 4 Sent Done Done false
29012 8851 Acked Read 4 Sent Done Done false
29013 8852 Acked Read 3 Sent Done Done false
29014 8853 Acked Write 8 Sent Done Done false
29015 8854 Acked Write 5 Sent Done Done false
29016 8855 Acked Write 4 Sent Done Done false
29017 8856 Acked Write 6 Sent Done Done false
29018 8857 Acked Write 6 Sent Done Done false
29019 8858 Acked Write 4 Sent Done Done false
29020 8859 Acked Read 10 Sent Done Done false
29021 8860 Acked Write 8 Sent Done Done false
29022 8861 Acked Read 5 Sent Done Done false
29023 STATES DS:0 DS:1 DS:2 TOTAL
29024 Sent 302 0 0 302
29025 Done 0 302 302 604
29026 Skipped 0 0 0 0
29027 Error 0 0 0 0
29028 Last Flush: 8559 8845 8845
29029 Downstairs last five completed: 8559 8558 8557 8556 8555
29030 Upstairs last five completed: 8861 8860 8859 8858 8857
29031 CLIENT: Up:0 ds:302 act:2
29032 [8810] got start action
29033 [8810] I am going to restart
29034 Starting downstairs at port 8810
29035 Make output file at "/tmp/dsc/downstairs-8810.txt"
29036 [8810][0] reports Starting
29037 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1225)
29038 [8810][0] reports Running
29039 ----------------------------------------------------------------
29040 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
29041 States: Offline Active Active
29042 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29043 8560 Acked Write 3 Sent Done Done false
29044 8561 Acked Write 10 Sent Done Done false
29045 8562 Acked Read 2 Sent Done Done false
29046 8563 Acked Read 1 Sent Done Done false
29047 8564 Acked Write 2 Sent Done Done false
29048 8565 Acked Flush 0 Sent Done Done false
29049 8566 Acked Read 2 Sent Done Done false
29050 8567 Acked Write 2 Sent Done Done false
29051 8568 Acked Read 2 Sent Done Done false
29052 8569 Acked Write 9 Sent Done Done false
29053 8570 Acked Write 8 Sent Done Done false
29054 8571 Acked Read 5 Sent Done Done false
29055 8572 Acked Read 5 Sent Done Done false
29056 8573 Acked Read 3 Sent Done Done false
29057 8574 Acked Read 9 Sent Done Done false
29058 8575 Acked Write 7 Sent Done Done false
29059 8576 Acked Write 4 Sent Done Done false
29060 8577 Acked Write 3 Sent Done Done false
29061 8578 Acked Read 7 Sent Done Done false
29062 8579 Acked Read 6 Sent Done Done false
29063 8580 Acked Flush 0 Sent Done Done false
29064 8581 Acked Flush 0 Sent Done Done false
29065 8582 Acked Write 8 Sent Done Done false
29066 8583 Acked Read 9 Sent Done Done false
29067 8584 Acked Read 3 Sent Done Done false
29068 8585 Acked Read 3 Sent Done Done false
29069 8586 Acked Write 7 Sent Done Done false
29070 8587 Acked Write 9 Sent Done Done false
29071 8588 Acked Write 9 Sent Done Done false
29072 8589 Acked Read 6 Sent Done Done false
29073 8590 Acked Write 4 Sent Done Done false
29074 8591 Acked Read 5 Sent Done Done false
29075 8592 Acked Write 9 Sent Done Done false
29076 8593 Acked Read 1 Sent Done Done false
29077 8594 Acked Flush 0 Sent Done Done false
29078 8595 Acked Write 1 Sent Done Done false
29079 8596 Acked Write 9 Sent Done Done false
29080 8597 Acked Flush 0 Sent Done Done false
29081 8598 Acked Flush 0 Sent Done Done false
29082 8599 Acked Read 7 Sent Done Done false
29083 8600 Acked Write 6 Sent Done Done false
29084 8601 Acked Flush 0 Sent Done Done false
29085 8602 Acked Read 8 Sent Done Done false
29086 8603 Acked Write 8 Sent Done Done false
29087 8604 Acked Write 5 Sent Done Done false
29088 8605 Acked Write 1 Sent Done Done false
29089 8606 Acked Write 5 Sent Done Done false
29090 8607 Acked Read 2 Sent Done Done false
29091 8608 Acked Read 3 Sent Done Done false
29092 8609 Acked Read 3 Sent Done Done false
29093 8610 Acked Read 5 Sent Done Done false
29094 8611 Acked Write 5 Sent Done Done false
29095 8612 Acked Flush 0 Sent Done Done false
29096 8613 Acked Flush 0 Sent Done Done false
29097 8614 Acked Write 6 Sent Done Done false
29098 8615 Acked Write 3 Sent Done Done false
29099 8616 Acked Write 1 Sent Done Done false
29100 8617 Acked Read 10 Sent Done Done false
29101 8618 Acked Read 2 Sent Done Done false
29102 8619 Acked Write 5 Sent Done Done false
29103 8620 Acked Write 5 Sent Done Done false
29104 8621 Acked Write 2 Sent Done Done false
29105 8622 Acked Write 7 Sent Done Done false
29106 8623 Acked Read 1 Sent Done Done false
29107 8624 Acked Write 3 Sent Done Done false
29108 8625 Acked Write 7 Sent Done Done false
29109 8626 Acked Write 7 Sent Done Done false
29110 8627 Acked Write 8 Sent Done Done false
29111 8628 Acked Write 3 Sent Done Done false
29112 8629 Acked Read 1 Sent Done Done false
29113 8630 Acked Write 6 Sent Done Done false
29114 8631 Acked Write 10 Sent Done Done false
29115 8632 Acked Write 8 Sent Done Done false
29116 8633 Acked Write 8 Sent Done Done false
29117 8634 Acked Write 4 Sent Done Done false
29118 8635 Acked Flush 0 Sent Done Done false
29119 8636 Acked Write 7 Sent Done Done false
29120 8637 Acked Flush 0 Sent Done Done false
29121 8638 Acked Flush 0 Sent Done Done false
29122 8639 Acked Read 2 Sent Done Done false
29123 8640 Acked Read 8 Sent Done Done false
29124 8641 Acked Read 4 Sent Done Done false
29125 8642 Acked Write 9 Sent Done Done false
29126 8643 Acked Read 1 Sent Done Done false
29127 8644 Acked Write 4 Sent Done Done false
29128 8645 Acked Read 4 Sent Done Done false
29129 8646 Acked Write 3 Sent Done Done false
29130 8647 Acked Read 1 Sent Done Done false
29131 8648 Acked Read 6 Sent Done Done false
29132 8649 Acked Write 4 Sent Done Done false
29133 8650 Acked Write 2 Sent Done Done false
29134 8651 Acked Read 10 Sent Done Done false
29135 8652 Acked Read 3 Sent Done Done false
29136 8653 Acked Read 1 Sent Done Done false
29137 8654 Acked Read 9 Sent Done Done false
29138 8655 Acked Write 2 Sent Done Done false
29139 8656 Acked Read 4 Sent Done Done false
29140 8657 Acked Write 4 Sent Done Done false
29141 8658 Acked Write 6 Sent Done Done false
29142 8659 Acked Write 9 Sent Done Done false
29143 8660 Acked Read 3 Sent Done Done false
29144 8661 Acked Write 8 Sent Done Done false
29145 8662 Acked Write 7 Sent Done Done false
29146 8663 Acked Read 7 Sent Done Done false
29147 8664 Acked Write 10 Sent Done Done false
29148 8665 Acked Write 3 Sent Done Done false
29149 8666 Acked Write 7 Sent Done Done false
29150 8667 Acked Write 4 Sent Done Done false
29151 8668 Acked Write 5 Sent Done Done false
29152 8669 Acked Flush 0 Sent Done Done false
29153 8670 Acked Read 7 Sent Done Done false
29154 8671 Acked Write 8 Sent Done Done false
29155 8672 Acked Read 10 Sent Done Done false
29156 8673 Acked Write 3 Sent Done Done false
29157 8674 Acked Read 8 Sent Done Done false
29158 8675 Acked Read 1 Sent Done Done false
29159 8676 Acked Read 6 Sent Done Done false
29160 8677 Acked Write 8 Sent Done Done false
29161 8678 Acked Flush 0 Sent Done Done false
29162 8679 Acked Write 6 Sent Done Done false
29163 8680 Acked Read 2 Sent Done Done false
29164 8681 Acked Read 7 Sent Done Done false
29165 8682 Acked Read 5 Sent Done Done false
29166 8683 Acked Write 5 Sent Done Done false
29167 8684 Acked Read 10 Sent Done Done false
29168 8685 Acked Flush 0 Sent Done Done false
29169 8686 Acked Read 7 Sent Done Done false
29170 8687 Acked Write 6 Sent Done Done false
29171 8688 Acked Flush 0 Sent Done Done false
29172 8689 Acked Flush 0 Sent Done Done false
29173 8690 Acked Flush 0 Sent Done Done false
29174 8691 Acked Flush 0 Sent Done Done false
29175 8692 Acked Read 7 Sent Done Done false
29176 8693 Acked Write 2 Sent Done Done false
29177 8694 Acked Write 7 Sent Done Done false
29178 8695 Acked Read 1 Sent Done Done false
29179 8696 Acked Write 10 Sent Done Done false
29180 8697 Acked Read 8 Sent Done Done false
29181 8698 Acked Read 10 Sent Done Done false
29182 8699 Acked Write 1 Sent Done Done false
29183 8700 Acked Read 6 Sent Done Done false
29184 8701 Acked Write 8 Sent Done Done false
29185 8702 Acked Read 6 Sent Done Done false
29186 8703 Acked Write 9 Sent Done Done false
29187 8704 Acked Read 8 Sent Done Done false
29188 8705 Acked Write 3 Sent Done Done false
29189 8706 Acked Read 7 Sent Done Done false
29190 8707 Acked Write 2 Sent Done Done false
29191 8708 Acked Flush 0 Sent Done Done false
29192 8709 Acked Write 6 Sent Done Done false
29193 8710 Acked Read 9 Sent Done Done false
29194 8711 Acked Read 10 Sent Done Done false
29195 8712 Acked Flush 0 Sent Done Done false
29196 8713 Acked Read 5 Sent Done Done false
29197 8714 Acked Read 2 Sent Done Done false
29198 8715 Acked Write 9 Sent Done Done false
29199 8716 Acked Read 2 Sent Done Done false
29200 8717 Acked Read 4 Sent Done Done false
29201 8718 Acked Read 3 Sent Done Done false
29202 8719 Acked Write 6 Sent Done Done false
29203 8720 Acked Read 9 Sent Done Done false
29204 8721 Acked Write 8 Sent Done Done false
29205 8722 Acked Write 4 Sent Done Done false
29206 8723 Acked Read 7 Sent Done Done false
29207 8724 Acked Read 5 Sent Done Done false
29208 8725 Acked Flush 0 Sent Done Done false
29209 8726 Acked Write 3 Sent Done Done false
29210 8727 Acked Read 2 Sent Done Done false
29211 8728 Acked Read 5 Sent Done Done false
29212 8729 Acked Read 10 Sent Done Done false
29213 8730 Acked Read 10 Sent Done Done false
29214 8731 Acked Read 10 Sent Done Done false
29215 8732 Acked Flush 0 Sent Done Done false
29216 8733 Acked Write 10 Sent Done Done false
29217 8734 Acked Read 7 Sent Done Done false
29218 8735 Acked Read 9 Sent Done Done false
29219 8736 Acked Write 7 Sent Done Done false
29220 8737 Acked Write 7 Sent Done Done false
29221 8738 Acked Read 6 Sent Done Done false
29222 8739 Acked Read 6 Sent Done Done false
29223 8740 Acked Read 2 Sent Done Done false
29224 8741 Acked Read 1 Sent Done Done false
29225 8742 Acked Write 3 Sent Done Done false
29226 8743 Acked Write 4 Sent Done Done false
29227 8744 Acked Read 10 Sent Done Done false
29228 8745 Acked Write 2 Sent Done Done false
29229 8746 Acked Read 10 Sent Done Done false
29230 8747 Acked Write 4 Sent Done Done false
29231 8748 Acked Read 9 Sent Done Done false
29232 8749 Acked Read 7 Sent Done Done false
29233 8750 Acked Write 1 Sent Done Done false
29234 8751 Acked Read 7 Sent Done Done false
29235 8752 Acked Read 10 Sent Done Done false
29236 8753 Acked Write 4 Sent Done Done false
29237 8754 Acked Write 3 Sent Done Done false
29238 8755 Acked Read 1 Sent Done Done false
29239 8756 Acked Write 2 Sent Done Done false
29240 8757 Acked Read 3 Sent Done Done false
29241 8758 Acked Read 3 Sent Done Done false
29242 8759 Acked Write 9 Sent Done Done false
29243 8760 Acked Flush 0 Sent Done Done false
29244 8761 Acked Read 4 Sent Done Done false
29245 8762 Acked Flush 0 Sent Done Done false
29246 8763 Acked Write 4 Sent Done Done false
29247 8764 Acked Write 3 Sent Done Done false
29248 8765 Acked Read 5 Sent Done Done false
29249 8766 Acked Write 3 Sent Done Done false
29250 8767 Acked Write 3 Sent Done Done false
29251 8768 Acked Write 5 Sent Done Done false
29252 8769 Acked Write 8 Sent Done Done false
29253 8770 Acked Write 2 Sent Done Done false
29254 8771 Acked Write 6 Sent Done Done false
29255 8772 Acked Read 9 Sent Done Done false
29256 8773 Acked Read 3 Sent Done Done false
29257 8774 Acked Flush 0 Sent Done Done false
29258 8775 Acked Write 4 Sent Done Done false
29259 8776 Acked Read 6 Sent Done Done false
29260 8777 Acked Read 4 Sent Done Done false
29261 8778 Acked Read 8 Sent Done Done false
29262 8779 Acked Write 5 Sent Done Done false
29263 8780 Acked Write 1 Sent Done Done false
29264 8781 Acked Read 10 Sent Done Done false
29265 8782 Acked Write 8 Sent Done Done false
29266 8783 Acked Read 1 Sent Done Done false
29267 8784 Acked Write 4 Sent Done Done false
29268 8785 Acked Write 9 Sent Done Done false
29269 8786 Acked Read 4 Sent Done Done false
29270 8787 Acked Write 2 Sent Done Done false
29271 8788 Acked Write 10 Sent Done Done false
29272 8789 Acked Read 3 Sent Done Done false
29273 8790 Acked Flush 0 Sent Done Done false
29274 8791 Acked Flush 0 Sent Done Done false
29275 8792 Acked Read 4 Sent Done Done false
29276 8793 Acked Write 3 Sent Done Done false
29277 8794 Acked Read 2 Sent Done Done false
29278 8795 Acked Read 9 Sent Done Done false
29279 8796 Acked Flush 0 Sent Done Done false
29280 8797 Acked Write 10 Sent Done Done false
29281 8798 Acked Read 8 Sent Done Done false
29282 8799 Acked Write 9 Sent Done Done false
29283 8800 Acked Flush 0 Sent Done Done false
29284 8801 Acked Read 10 Sent Done Done false
29285 8802 Acked Read 8 Sent Done Done false
29286 8803 Acked Read 1 Sent Done Done false
29287 8804 Acked Read 2 Sent Done Done false
29288 8805 Acked Write 7 Sent Done Done false
29289 8806 Acked Write 7 Sent Done Done false
29290 8807 Acked Write 6 Sent Done Done false
29291 8808 Acked Write 2 Sent Done Done false
29292 8809 Acked Read 4 Sent Done Done false
29293 8810 Acked Write 9 Sent Done Done false
29294 8811 Acked Read 8 Sent Done Done false
29295 8812 Acked Write 2 Sent Done Done false
29296 8813 Acked Write 4 Sent Done Done false
29297 8814 Acked Write 7 Sent Done Done false
29298 8815 Acked Flush 0 Sent Done Done false
29299 8816 Acked Read 8 Sent Done Done false
29300 8817 Acked Flush 0 Sent Done Done false
29301 8818 Acked Read 9 Sent Done Done false
29302 8819 Acked Write 1 Sent Done Done false
29303 8820 Acked Flush 0 Sent Done Done false
29304 8821 Acked Read 8 Sent Done Done false
29305 8822 Acked Write 10 Sent Done Done false
29306 8823 Acked Write 9 Sent Done Done false
29307 8824 Acked Read 1 Sent Done Done false
29308 8825 Acked Write 4 Sent Done Done false
29309 8826 Acked Read 7 Sent Done Done false
29310 8827 Acked Read 10 Sent Done Done false
29311 8828 Acked Read 6 Sent Done Done false
29312 8829 Acked Write 7 Sent Done Done false
29313 8830 Acked Read 9 Sent Done Done false
29314 8831 Acked Write 5 Sent Done Done false
29315 8832 Acked Write 2 Sent Done Done false
29316 8833 Acked Write 3 Sent Done Done false
29317 8834 Acked Flush 0 Sent Done Done false
29318 8835 Acked Read 7 Sent Done Done false
29319 8836 Acked Flush 0 Sent Done Done false
29320 8837 Acked Read 8 Sent Done Done false
29321 8838 Acked Write 9 Sent Done Done false
29322 8839 Acked Write 6 Sent Done Done false
29323 8840 Acked Write 10 Sent Done Done false
29324 8841 Acked Write 9 Sent Done Done false
29325 8842 Acked Read 6 Sent Done Done false
29326 8843 Acked Write 3 Sent Done Done false
29327 8844 Acked Read 10 Sent Done Done false
29328 8845 Acked Flush 0 Sent Done Done false
29329 8846 Acked Write 3 Sent Done Done false
29330 8847 Acked Write 4 Sent Done Done false
29331 8848 Acked Read 4 Sent Done Done false
29332 8849 Acked Read 5 Sent Done Done false
29333 8850 Acked Read 4 Sent Done Done false
29334 8851 Acked Read 4 Sent Done Done false
29335 8852 Acked Read 3 Sent Done Done false
29336 8853 Acked Write 8 Sent Done Done false
29337 8854 Acked Write 5 Sent Done Done false
29338 8855 Acked Write 4 Sent Done Done false
29339 8856 Acked Write 6 Sent Done Done false
29340 8857 Acked Write 6 Sent Done Done false
29341 8858 Acked Write 4 Sent Done Done false
29342 8859 Acked Read 10 Sent Done Done false
29343 8860 Acked Write 8 Sent Done Done false
29344 8861 Acked Read 5 Sent Done Done false
29345 8862 Acked Flush 0 Sent Done Done false
29346 STATES DS:0 DS:1 DS:2 TOTAL
29347 Sent 303 0 0 303
29348 Done 0 303 303 606
29349 Skipped 0 0 0 0
29350 Error 0 0 0 0
29351 Last Flush: 8559 8862 8862
29352 Downstairs last five completed: 8559 8558 8557 8556 8555
29353 Upstairs last five completed: 8862 8861 8860 8859 8858
29354 CLIENT: Up:0 ds:303 act:2
293552024-11-07T14:09:27.204ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293562024-11-07T14:09:27.204ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58841) = io task client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293572024-11-07T14:09:27.205ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293582024-11-07T14:09:27.205ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293592024-11-07T14:09:27.205ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293602024-11-07T14:09:27.205ZINFOcrucible: send last flush ID to this DS: 8559 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293612024-11-07T14:09:27.206ZINFOcrucible: Replied this last flush ID: 8559 = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293622024-11-07T14:09:27.206ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293632024-11-07T14:09:27.206ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293642024-11-07T14:09:27.206ZINFOcrucible: [0] client re-new 303 jobs since flush 8559 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293652024-11-07T14:09:27.206ZINFOcrucible: [0] Replayed 303 jobs since flush: 8559 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
29366 ----------------------------------------------------------------
29367 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29368 Downstairs last five completed: 8862 8861 8860 8859 8858
29369 Upstairs last five completed: 8862 8861 8860 8859 8858
29370 CLIENT: Up:0 ds:0 act:3
29371 Replay: All jobs finished
29372 Main task has work to do, go find it
29373 got dsc Stop(1)
29374 stop 1
29375 [8820] Got stop action so:false kr:false
29376 Nov 07 14:09:29.495 INFO request completed, latency_us: 344, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 7a1e7453-d027-4c2b-a250-5f240ae47111, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
293772024-11-07T14:09:29.496ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293782024-11-07T14:09:29.496ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293792024-11-07T14:09:29.496ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293802024-11-07T14:09:29.496ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293812024-11-07T14:09:29.496ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
293822024-11-07T14:09:29.496ZINFOcrucible: client task is exiting = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
29383 Nov 07 14:09:29.496 INFO request completed, latency_us: 161, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4c82392a-3c9a-452f-a74b-184872929585, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
29384 [8820] Exited with: ExitStatus(unix_wait_status(9))
29385 [8820][1] reports Exit
29386 Nov 07 14:09:33.498 INFO request completed, latency_us: 236, response_code: 200, uri: /state/cid/1, method: GET, req_id: d0e71ba5-8224-4bcb-9a3a-c847b429bcee, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
29387 001/300 Write block 556 len 24576 data: 5 7 6 6 7 7
29388 002/300 Read block 2656 len 36864
29389 003/300 Read block 2921 len 24576
29390 004/300 Write block 2039 len 12288 data: 3 4 4
29391 005/300 Flush
29392 006/300 Read block 3284 len 24576
29393 007/300 Write block 2940 len 4096 data: 7
29394 008/300 Write block 522 len 40960 data: 4 4 3 3 3 7 6 6 5 7
29395 009/300 Flush
29396 010/300 Flush
29397 011/300 Read block 1338 len 16384
29398 012/300 Read block 3623 len 40960
29399 013/300 Read block 1289 len 36864
29400 014/300 Read block 3304 len 8192
29401 015/300 Flush
29402 016/300 Write block 427 len 24576 data: 5 8 8 7 6 6
29403 017/300 Read block 4184 len 28672
29404 018/300 Write block 3839 len 12288 data: 5 5 5
29405 019/300 Write block 359 len 40960 data: 3 4 4 6 5 4 4 4 4 5
29406 020/300 Flush
29407 021/300 Read block 1128 len 32768
29408 022/300 Read block 1051 len 40960
29409 023/300 Read block 1154 len 40960
29410 024/300 Write block 1016 len 16384 data: 5 5 4 4
29411 025/300 Write block 2821 len 20480 data: 7 6 7 8 7
29412 026/300 Read block 2085 len 36864
29413 027/300 Write block 3668 len 28672 data: 3 4 3 2 4 4 7
29414 028/300 Read block 4216 len 8192
29415 029/300 Read block 663 len 32768
29416 030/300 Write block 3938 len 24576 data: 5 4 4 3 3 3
29417 031/300 Write block 3900 len 28672 data: 5 5 5 7 7 5 5
29418 032/300 Flush
29419 033/300 Flush
29420 034/300 Flush
29421 035/300 Read block 4383 len 4096
29422 036/300 Flush
29423 037/300 Read block 4414 len 28672
29424 038/300 Write block 2794 len 36864 data: 8 6 7 6 6 4 4 6 5
29425 039/300 Read block 1396 len 32768
29426 040/300 Flush
29427 041/300 Read block 1084 len 12288
29428 042/300 Read block 3707 len 4096
29429 043/300 Write block 1490 len 12288 data: 5 5 5
29430 044/300 Write block 2497 len 32768 data: 5 5 6 5 4 4 3 4
29431 045/300 Read block 4687 len 36864
29432 046/300 Read block 3076 len 40960
29433 047/300 Read block 1777 len 16384
29434 048/300 Read block 4287 len 4096
29435 049/300 Read block 4557 len 28672
29436 050/300 Read block 4635 len 32768
29437 051/300 Read block 4468 len 36864
29438 052/300 Write block 2262 len 28672 data: 2 2 3 3 3 3 4
29439 053/300 Read block 2187 len 20480
29440 054/300 Flush
29441 055/300 Read block 1901 len 24576
29442 056/300 Flush
29443 057/300 Read block 1696 len 40960
29444 058/300 Read block 1816 len 4096
29445 059/300 Read block 691 len 24576
29446 060/300 Write block 2255 len 36864 data: 3 3 4 4 4 3 2 3 3
29447 061/300 Write block 990 len 28672 data: 3 3 4 4 5 5 6
29448 062/300 Read block 1995 len 24576
29449 063/300 Read block 4112 len 32768
29450 064/300 Write block 781 len 40960 data: 3 3 2 2 3 3 3 3 3 5
29451 065/300 Read block 810 len 8192
29452 066/300 Read block 1391 len 16384
29453 067/300 Write block 971 len 28672 data: 6 6 6 5 6 7 7
29454 068/300 Write block 484 len 40960 data: 7 8 7 8 8 4 3 3 3 3
29455 069/300 Read block 3823 len 4096
29456 070/300 Write block 2983 len 4096 data: 4
29457 071/300 Read block 4768 len 28672
29458 072/300 Write block 4334 len 40960 data: 6 4 4 6 6 5 6 5 5 3
29459 073/300 Read block 1610 len 36864
29460 074/300 Write block 306 len 36864 data: 5 6 6 5 4 2 4 4 4
29461 075/300 Write block 3929 len 40960 data: 4 4 4 4 4 4 5 3 4 6
29462 076/300 Read block 949 len 4096
29463 077/300 Read block 4513 len 4096
29464 078/300 Read block 3715 len 32768
29465 079/300 Write block 1127 len 40960 data: 5 4 5 5 5 4 4 6 6 5
29466 080/300 Read block 1217 len 20480
29467 081/300 Read block 374 len 28672
29468 082/300 Write block 2183 len 4096 data: 6
29469 083/300 Read block 3239 len 4096
29470 084/300 Read block 3918 len 28672
29471 085/300 Write block 307 len 4096 data: 7
29472 086/300 Write block 4920 len 4096 data: 9
29473 087/300 Flush
29474 088/300 Flush
29475 089/300 Read block 1234 len 16384
29476 090/300 Write block 2749 len 36864 data: 4 5 4 5 6 7 6 6 6
29477 091/300 Read block 1548 len 36864
29478 092/300 Read block 4705 len 40960
29479 093/300 Write block 1409 len 16384 data: 5 5 3 4
29480 094/300 Read block 262 len 40960
29481 095/300 Write block 1088 len 16384 data: 6 8 5 3
29482 096/300 Write block 3056 len 12288 data: 6 4 6
29483 097/300 Write block 179 len 40960 data: 6 6 7 8 6 7 6 8 7 7
29484 098/300 Flush
29485 099/300 Read block 2167 len 20480
29486 100/300 Write block 3469 len 20480 data: 4 4 4 5 5
29487 101/300 Read block 3787 len 28672
29488 102/300 Flush
29489 103/300 Write block 2213 len 24576 data: 4 4 3 3 3 3
29490 104/300 Write block 157 len 16384 data: 6 6 8 9
29491 105/300 Write block 4629 len 28672 data: 5 4 4 4 4 4 5
29492 106/300 Read block 1002 len 20480
29493 107/300 Read block 1179 len 16384
29494 108/300 Write block 4081 len 36864 data: 4 3 3 4 5 5 6 6 5
29495 109/300 Write block 2748 len 24576 data: 4 5 6 5 6 7
29496 110/300 Write block 4532 len 12288 data: 9 8 9
29497 111/300 Write block 1658 len 20480 data: 5 7 8 8 6
29498 112/300 Write block 1116 len 16384 data: 5 5 6 6
29499 113/300 Write block 2670 len 4096 data: 8
29500 114/300 Read block 1224 len 40960
29501 115/300 Flush
29502 116/300 Read block 4066 len 8192
29503 117/300 Read block 4898 len 24576
29504 118/300 Read block 792 len 16384
29505 119/300 Write block 4859 len 4096 data: 2
29506 120/300 Write block 2407 len 24576 data: 4 4 4 5 5 5
29507 121/300 Read block 2940 len 28672
29508 122/300 Read block 3737 len 24576
29509 123/300 Write block 2033 len 20480 data: 5 5 3 4 3
29510 124/300 Read block 1886 len 20480
29511 125/300 Read block 4329 len 28672
29512 126/300 Flush
29513 127/300 Read block 1796 len 16384
29514 128/300 Read block 2866 len 36864
29515 129/300 Write block 119 len 32768 data: 4 4 4 5 5 6 6 6
29516 130/300 Flush
29517 131/300 Read block 2876 len 16384
29518 132/300 Read block 29 len 12288
29519 133/300 Read block 4448 len 32768
29520 134/300 Read block 2126 len 20480
29521 135/300 Write block 22 len 28672 data: 5 5 6 8 10 10 11
29522 136/300 Read block 2025 len 36864
29523 137/300 Write block 1556 len 8192 data: 4 5
29524 138/300 Read block 2544 len 4096
29525 139/300 Write block 3759 len 36864 data: 4 4 5 5 5 5 4 4 4
29526 140/300 Write block 4073 len 24576 data: 3 3 4 5 5 5
29527 141/300 Write block 1375 len 16384 data: 7 8 8 6
29528 142/300 Read block 1556 len 36864
29529 143/300 Write block 1802 len 12288 data: 5 6 6
29530 144/300 Read block 2203 len 32768
29531 145/300 Write block 4903 len 4096 data: 2
29532 146/300 Read block 899 len 40960
29533 147/300 Write block 4765 len 32768 data: 8 7 6 5 5 6 7 6
29534 148/300 Read block 4764 len 24576
29535 149/300 Write block 2887 len 36864 data: 7 7 5 3 4 4 4 4 4
29536 150/300 Read block 2819 len 32768
29537 151/300 Flush
29538 152/300 Write block 2056 len 40960 data: 4 5 4 6 6 6 4 4 5 4
29539 153/300 Write block 2816 len 4096 data: 6
29540 154/300 Read block 4848 len 36864
29541 155/300 Write block 3296 len 12288 data: 8 8 7
29542 156/300 Read block 586 len 28672
29543 157/300 Read block 2291 len 28672
29544 158/300 Write block 4543 len 40960 data: 6 6 5 4 4 3 4 4 3 3
29545 159/300 Write block 4560 len 16384 data: 6 4 5 5
29546 160/300 Write block 2171 len 16384 data: 6 5 3 3
29547 161/300 Write block 967 len 8192 data: 2 2
29548 162/300 Flush
29549 163/300 Write block 828 len 12288 data: 6 7 8
29550 164/300 Read block 3661 len 20480
29551 165/300 Read block 1899 len 32768
29552 166/300 Write block 2122 len 32768 data: 4 4 4 4 2 3 3 4
29553 167/300 Read block 2823 len 24576
29554 168/300 Flush
29555 169/300 Read block 529 len 28672
29556 170/300 Read block 2813 len 4096
29557 171/300 Write block 1584 len 40960 data: 5 5 6 7 5 5 5 5 5 5
29558 172/300 Read block 1142 len 4096
29559 173/300 Read block 2720 len 20480
29560 174/300 Write block 136 len 20480 data: 3 3 3 3 4
29561 175/300 Write block 812 len 4096 data: 4
29562 176/300 Write block 598 len 24576 data: 5 4 4 4 4 5
29563 177/300 Write block 4583 len 40960 data: 5 4 4 6 6 7 6 7 8 8
29564 178/300 Write block 4782 len 32768 data: 3 3 4 5 5 6 7 7
29565 179/300 Read block 789 len 36864
29566 180/300 Read block 3635 len 20480
29567 181/300 Read block 609 len 4096
29568 182/300 Read block 951 len 24576
29569 183/300 Flush
29570 184/300 Write block 2374 len 16384 data: 2 3 3 3
29571 185/300 Write block 463 len 28672 data: 5 5 6 7 8 7 7
29572 186/300 Flush
29573 187/300 Write block 4909 len 8192 data: 7 5
29574 188/300 Write block 2414 len 8192 data: 3 5
29575 189/300 Write block 2448 len 4096 data: 8
29576 190/300 Write block 777 len 36864 data: 4 4 3 3 4 4 3 3 4
29577 191/300 Write block 1353 len 20480 data: 7 9 8 12 11
29578 192/300 Read block 2197 len 36864
29579 193/300 Write block 1744 len 36864 data: 4 5 5 3 4 7 10 11 9
29580 194/300 Read block 3065 len 32768
29581 195/300 Read block 3051 len 40960
29582 196/300 Read block 1615 len 32768
29583 197/300 Read block 2753 len 12288
29584 198/300 Write block 2786 len 24576 data: 8 8 7 7 8 8
29585 199/300 Write block 1087 len 20480 data: 5 7 9 6 4
29586 200/300 Write block 3226 len 12288 data: 4 5 5
29587 201/300 Read block 2013 len 4096
29588 202/300 Read block 4542 len 8192
29589 203/300 Write block 876 len 16384 data: 5 6 7 6
29590 204/300 Read block 1690 len 40960
29591 205/300 Read block 1995 len 32768
29592 206/300 Read block 4300 len 16384
29593 207/300 Write block 2138 len 32768 data: 5 6 5 4 2 5 7 6
29594 208/300 Write block 987 len 28672 data: 6 6 6 4 4 5 5
29595 209/300 Write block 3983 len 40960 data: 3 2 5 5 5 5 5 4 6 5
29596 210/300 Write block 458 len 8192 data: 7 6
29597 211/300 Read block 1199 len 20480
29598 212/300 Flush
29599 213/300 Write block 745 len 20480 data: 6 5 3 3 3
29600 214/300 Write block 2038 len 40960 data: 3 4 5 5 4 5 5 5 3 3
29601 215/300 Read block 960 len 12288
29602 216/300 Write block 1426 len 40960 data: 4 3 4 5 5 6 7 8 7 5
29603 217/300 Flush
29604 218/300 Write block 1012 len 32768 data: 6 6 7 7 6 6 5 5
29605 219/300 Read block 1360 len 16384
29606 220/300 Write block 3864 len 24576 data: 10 8 7 6 6 7
29607 221/300 Write block 2883 len 40960 data: 8 7 7 6 8 8 6 4 5 5
29608 222/300 Write block 2915 len 36864 data: 6 6 6 6 6 6 6 5 3
29609 223/300 Read block 2556 len 36864
29610 224/300 Write block 2610 len 24576 data: 5 6 7 6 6 6
29611 225/300 Read block 1159 len 40960
29612 226/300 Read block 1612 len 20480
29613 227/300 Write block 2049 len 8192 data: 4 5
29614 228/300 Write block 2557 len 24576 data: 7 6 7 8 7 6
29615 229/300 Write block 4465 len 4096 data: 7
29616 230/300 Read block 618 len 8192
29617 231/300 Read block 594 len 28672
29618 232/300 Read block 728 len 40960
29619 233/300 Flush
29620 234/300 Read block 307 len 8192
29621 235/300 Read block 2771 len 40960
29622 236/300 Write block 1146 len 24576 data: 8 6 4 3 3 2
29623 237/300 Read block 3704 len 12288
29624 238/300 Read block 906 len 28672
29625 239/300 Read block 1637 len 12288
29626 240/300 Read block 1152 len 8192
29627 241/300 Flush
29628 242/300 Write block 569 len 12288 data: 8 6 5
29629 243/300 Read block 2642 len 40960
29630 244/300 Write block 2236 len 32768 data: 4 4 5 6 7 9 7 7
29631 245/300 Write block 1088 len 40960 data: 8 10 7 5 5 5 5 5 5 5
29632 246/300 Write block 1440 len 28672 data: 5 5 8 8 7 6 6
29633 247/300 Write block 4772 len 28672 data: 7 5 5 4 4 4 5
29634 248/300 Read block 27 len 40960
29635 249/300 Read block 387 len 24576
29636 250/300 Read block 1116 len 28672
29637 251/300 Read block 739 len 28672
29638 252/300 Read block 206 len 8192
29639 253/300 Read block 572 len 40960
29640 254/300 Read block 378 len 28672
29641 255/300 Write block 1123 len 32768 data: 5 5 5 6 6 5 6 6
29642 256/300 Read block 2650 len 36864
29643 257/300 Read block 3154 len 16384
29644 258/300 Read block 2960 len 32768
29645 259/300 Read block 3313 len 20480
29646 260/300 Flush
29647 261/300 Write block 3839 len 20480 data: 6 6 6 6 6
29648 262/300 Write block 3066 len 16384 data: 4 4 5 4
29649 263/300 Write block 4497 len 8192 data: 4 4
29650 264/300 Read block 3354 len 40960
29651 265/300 Write block 2609 len 12288 data: 5 6 7
29652 266/300 Write block 2026 len 36864 data: 6 5 6 6 5 5 6 6 6
29653 267/300 Write block 2732 len 8192 data: 9 8
29654 268/300 Read block 1548 len 28672
29655 269/300 Write block 311 len 20480 data: 3 5 5 5 3
29656 270/300 Write block 4802 len 36864 data: 6 6 3 3 3 4 3 4 4
29657 271/300 Read block 4474 len 24576
29658 272/300 Read block 293 len 32768
29659 273/300 Write block 752 len 8192 data: 6 5
29660 274/300 Write block 191 len 12288 data: 5 6 5
29661 275/300 Read block 3809 len 36864
29662 276/300 Write block 2119 len 4096 data: 5
29663 277/300 Flush
29664 278/300 Read block 4818 len 40960
29665 279/300 Write block 545 len 36864 data: 7 6 7 6 5 7 6 6 4
29666 280/300 Write block 956 len 24576 data: 7 7 5 5 3 3
29667 281/300 Read block 2909 len 20480
29668 282/300 Write block 2575 len 40960 data: 4 5 5 6 6 6 3 4 4 4
29669 283/300 Write block 3842 len 4096 data: 7
29670 284/300 Read block 3987 len 4096
29671 285/300 Write block 4440 len 32768 data: 9 10 9 9 8 7 6 6
29672 286/300 Write block 3588 len 4096 data: 5
29673 287/300 Read block 2996 len 4096
29674 288/300 Write block 1406 len 8192 data: 8 7
29675 289/300 Write block 775 len 28672 data: 6 6 5 5 4 4 5
29676 290/300 Read block 4608 len 36864
29677 291/300 Write block 450 len 32768 data: 7 7 10 8 9 8 10 10
29678 292/300 Read block 1841 len 12288
29679 293/300 Flush
29680 294/300 Flush
29681 295/300 Write block 445 len 12288 data: 8 6 5
29682 296/300 Read block 1453 len 36864
29683 297/300 Read block 1134 len 24576
29684 298/300 Write block 3041 len 8192 data: 3 3
29685 299/300 Write block 4912 len 32768 data: 6 6 6 7 6 6 7 9
29686 300/300 Write block 4474 len 12288 data: 4 5 5
29687 301/300 Write block 1353 len 24576 data: 8 10 9 13 12 11
29688 Main task has work to do, go find it
29689 got dsc Start(1)
29690 start 1
29691 Nov 07 14:09:33.801 INFO request completed, latency_us: 239, response_code: 204, uri: /start/cid/1, method: POST, req_id: f60ecff8-9ec5-4584-99da-3ee00921fc42, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
29692 Replay: started 1, returned:Ok(())
29693 ----------------------------------------------------------------
29694 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29695 States: Active Offline Active
29696 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29697 8863 Acked Write 6 Done Sent Done false
29698 8864 Acked Read 9 Done Sent Done false
29699 8865 Acked Read 6 Done Sent Done false
29700 8866 Acked Flush 0 Done Sent Done false
29701 8867 Acked Write 3 Done Sent Done false
29702 8868 Acked Flush 0 Done Sent Done false
29703 8869 Acked Read 6 Done Sent Done false
29704 8870 Acked Write 1 Done Sent Done false
29705 8871 Acked Write 10 Done Sent Done false
29706 8872 Acked Flush 0 Done Sent Done false
29707 8873 Acked Flush 0 Done Sent Done false
29708 8874 Acked Read 4 Done Sent Done false
29709 8875 Acked Read 10 Done Sent Done false
29710 8876 Acked Read 9 Done Sent Done false
29711 8877 Acked Read 2 Done Sent Done false
29712 8878 Acked Flush 0 Done Sent Done false
29713 8879 Acked Write 6 Done Sent Done false
29714 8880 Acked Read 7 Done Sent Done false
29715 8881 Acked Write 3 Done Sent Done false
29716 8882 Acked Write 10 Done Sent Done false
29717 8883 Acked Flush 0 Done Sent Done false
29718 8884 Acked Read 8 Done Sent Done false
29719 8885 Acked Read 10 Done Sent Done false
29720 8886 Acked Read 10 Done Sent Done false
29721 8887 Acked Write 4 Done Sent Done false
29722 8888 Acked Write 5 Done Sent Done false
29723 8889 Acked Read 9 Done Sent Done false
29724 8890 Acked Write 7 Done Sent Done false
29725 8891 Acked Read 2 Done Sent Done false
29726 8892 Acked Read 8 Done Sent Done false
29727 8893 Acked Write 6 Done Sent Done false
29728 8894 Acked Write 7 Done Sent Done false
29729 8895 Acked Flush 0 Done Sent Done false
29730 8896 Acked Flush 0 Done Sent Done false
29731 8897 Acked Flush 0 Done Sent Done false
29732 8898 Acked Read 1 Done Sent Done false
29733 8899 Acked Flush 0 Done Sent Done false
29734 8900 Acked Read 7 Done Sent Done false
29735 8901 Acked Write 9 Done Sent Done false
29736 8902 Acked Read 8 Done Sent Done false
29737 8903 Acked Flush 0 Done Sent Done false
29738 8904 Acked Read 3 Done Sent Done false
29739 8905 Acked Read 1 Done Sent Done false
29740 8906 Acked Write 3 Done Sent Done false
29741 8907 Acked Write 8 Done Sent Done false
29742 8908 Acked Read 9 Done Sent Done false
29743 8909 Acked Read 10 Done Sent Done false
29744 8910 Acked Read 4 Done Sent Done false
29745 8911 Acked Read 1 Done Sent Done false
29746 8912 Acked Read 7 Done Sent Done false
29747 8913 Acked Read 8 Done Sent Done false
29748 8914 Acked Read 9 Done Sent Done false
29749 8915 Acked Write 7 Done Sent Done false
29750 8916 Acked Read 5 Done Sent Done false
29751 8917 Acked Flush 0 Done Sent Done false
29752 8918 Acked Read 6 Done Sent Done false
29753 8919 Acked Flush 0 Done Sent Done false
29754 8920 Acked Read 10 Done Sent Done false
29755 8921 Acked Read 1 Done Sent Done false
29756 8922 Acked Read 6 Done Sent Done false
29757 8923 Acked Write 9 Done Sent Done false
29758 8924 Acked Write 7 Done Sent Done false
29759 8925 Acked Read 6 Done Sent Done false
29760 8926 Acked Read 8 Done Sent Done false
29761 8927 Acked Write 10 Done Sent Done false
29762 8928 Acked Read 2 Done Sent Done false
29763 8929 Acked Read 4 Done Sent Done false
29764 8930 Acked Write 7 Done Sent Done false
29765 8931 Acked Write 10 Done Sent Done false
29766 8932 Acked Read 1 Done Sent Done false
29767 8933 Acked Write 1 Done Sent Done false
29768 8934 Acked Read 7 Done Sent Done false
29769 8935 Acked Write 10 Done Sent Done false
29770 8936 Acked Read 9 Done Sent Done false
29771 8937 Acked Write 9 Done Sent Done false
29772 8938 Acked Write 10 Done Sent Done false
29773 8939 Acked Read 1 Done Sent Done false
29774 8940 Acked Read 1 Done Sent Done false
29775 8941 Acked Read 8 Done Sent Done false
29776 8942 Acked Write 10 Done Sent Done false
29777 8943 Acked Read 5 Done Sent Done false
29778 8944 Acked Read 7 Done Sent Done false
29779 8945 Acked Write 1 Done Sent Done false
29780 8946 Acked Read 1 Done Sent Done false
29781 8947 Acked Read 7 Done Sent Done false
29782 8948 Acked Write 1 Done Sent Done false
29783 8949 Acked Write 1 Done Sent Done false
29784 8950 Acked Flush 0 Done Sent Done false
29785 8951 Acked Flush 0 Done Sent Done false
29786 8952 Acked Read 4 Done Sent Done false
29787 8953 Acked Write 9 Done Sent Done false
29788 8954 Acked Read 9 Done Sent Done false
29789 8955 Acked Read 10 Done Sent Done false
29790 8956 Acked Write 4 Done Sent Done false
29791 8957 Acked Read 10 Done Sent Done false
29792 8958 Acked Write 4 Done Sent Done false
29793 8959 Acked Write 3 Done Sent Done false
29794 8960 Acked Write 10 Done Sent Done false
29795 8961 Acked Flush 0 Done Sent Done false
29796 8962 Acked Read 5 Done Sent Done false
29797 8963 Acked Write 5 Done Sent Done false
29798 8964 Acked Read 7 Done Sent Done false
29799 8965 Acked Flush 0 Done Sent Done false
29800 8966 Acked Write 6 Done Sent Done false
29801 8967 Acked Write 4 Done Sent Done false
29802 8968 Acked Write 7 Done Sent Done false
29803 8969 Acked Read 5 Done Sent Done false
29804 8970 Acked Read 4 Done Sent Done false
29805 8971 Acked Write 9 Done Sent Done false
29806 8972 Acked Write 6 Done Sent Done false
29807 8973 Acked Write 3 Done Sent Done false
29808 8974 Acked Write 5 Done Sent Done false
29809 8975 Acked Write 4 Done Sent Done false
29810 8976 Acked Write 1 Done Sent Done false
29811 8977 Acked Read 10 Done Sent Done false
29812 8978 Acked Flush 0 Done Sent Done false
29813 8979 Acked Read 2 Done Sent Done false
29814 8980 Acked Read 6 Done Sent Done false
29815 8981 Acked Read 4 Done Sent Done false
29816 8982 Acked Write 1 Done Sent Done false
29817 8983 Acked Write 6 Done Sent Done false
29818 8984 Acked Read 7 Done Sent Done false
29819 8985 Acked Read 6 Done Sent Done false
29820 8986 Acked Write 5 Done Sent Done false
29821 8987 Acked Read 5 Done Sent Done false
29822 8988 Acked Read 7 Done Sent Done false
29823 8989 Acked Flush 0 Done Sent Done false
29824 8990 Acked Read 4 Done Sent Done false
29825 8991 Acked Read 9 Done Sent Done false
29826 8992 Acked Write 8 Done Sent Done false
29827 8993 Acked Flush 0 Done Sent Done false
29828 8994 Acked Read 4 Done Sent Done false
29829 8995 Acked Read 3 Done Sent Done false
29830 8996 Acked Read 8 Done Sent Done false
29831 8997 Acked Read 5 Done Sent Done false
29832 8998 Acked Write 7 Done Sent Done false
29833 8999 Acked Read 9 Done Sent Done false
29834 9000 Acked Write 2 Done Sent Done false
29835 9001 Acked Read 1 Done Sent Done false
29836 9002 Acked Write 9 Done Sent Done false
29837 9003 Acked Write 6 Done Sent Done false
29838 9004 Acked Write 4 Done Sent Done false
29839 9005 Acked Read 9 Done Sent Done false
29840 9006 Acked Write 3 Done Sent Done false
29841 9007 Acked Read 8 Done Sent Done false
29842 9008 Acked Write 1 Done Sent Done false
29843 9009 Acked Read 10 Done Sent Done false
29844 9010 Acked Write 8 Done Sent Done false
29845 9011 Acked Read 6 Done Sent Done false
29846 9012 Acked Write 9 Done Sent Done false
29847 9013 Acked Read 8 Done Sent Done false
29848 9014 Acked Flush 0 Done Sent Done false
29849 9015 Acked Write 10 Done Sent Done false
29850 9016 Acked Write 1 Done Sent Done false
29851 9017 Acked Read 9 Done Sent Done false
29852 9018 Acked Write 3 Done Sent Done false
29853 9019 Acked Read 7 Done Sent Done false
29854 9020 Acked Read 7 Done Sent Done false
29855 9021 Acked Write 10 Done Sent Done false
29856 9022 Acked Write 4 Done Sent Done false
29857 9023 Acked Write 4 Done Sent Done false
29858 9024 Acked Write 2 Done Sent Done false
29859 9025 Acked Flush 0 Done Sent Done false
29860 9026 Acked Write 3 Done Sent Done false
29861 9027 Acked Read 5 Done Sent Done false
29862 9028 Acked Read 8 Done Sent Done false
29863 9029 Acked Write 8 Done Sent Done false
29864 9030 Acked Read 6 Done Sent Done false
29865 9031 Acked Flush 0 Done Sent Done false
29866 9032 Acked Read 7 Done Sent Done false
29867 9033 Acked Read 1 Done Sent Done false
29868 9034 Acked Write 10 Done Sent Done false
29869 9035 Acked Read 1 Done Sent Done false
29870 9036 Acked Read 5 Done Sent Done false
29871 9037 Acked Write 5 Done Sent Done false
29872 9038 Acked Write 1 Done Sent Done false
29873 9039 Acked Write 6 Done Sent Done false
29874 9040 Acked Write 10 Done Sent Done false
29875 9041 Acked Write 8 Done Sent Done false
29876 9042 Acked Read 9 Done Sent Done false
29877 9043 Acked Read 5 Done Sent Done false
29878 9044 Acked Read 1 Done Sent Done false
29879 9045 Acked Read 6 Done Sent Done false
29880 9046 Acked Flush 0 Done Sent Done false
29881 9047 Acked Write 4 Done Sent Done false
29882 9048 Acked Write 7 Done Sent Done false
29883 9049 Acked Flush 0 Done Sent Done false
29884 9050 Acked Write 2 Done Sent Done false
29885 9051 Acked Write 2 Done Sent Done false
29886 9052 Acked Write 1 Done Sent Done false
29887 9053 Acked Write 9 Done Sent Done false
29888 9054 Acked Write 5 Done Sent Done false
29889 9055 Acked Read 9 Done Sent Done false
29890 9056 Acked Write 9 Done Sent Done false
29891 9057 Acked Read 8 Done Sent Done false
29892 9058 Acked Read 10 Done Sent Done false
29893 9059 Acked Read 8 Done Sent Done false
29894 9060 Acked Read 3 Done Sent Done false
29895 9061 Acked Write 6 Done Sent Done false
29896 9062 Acked Write 5 Done Sent Done false
29897 9063 Acked Write 3 Done Sent Done false
29898 9064 Acked Read 1 Done Sent Done false
29899 9065 Acked Read 2 Done Sent Done false
29900 9066 Acked Write 4 Done Sent Done false
29901 9067 Acked Read 10 Done Sent Done false
29902 9068 Acked Read 8 Done Sent Done false
29903 9069 Acked Read 4 Done Sent Done false
29904 9070 Acked Write 8 Done Sent Done false
29905 9071 Acked Write 7 Done Sent Done false
29906 9072 Acked Write 10 Done Sent Done false
29907 9073 Acked Write 2 Done Sent Done false
29908 9074 Acked Read 5 Done Sent Done false
29909 9075 Acked Flush 0 Done Sent Done false
29910 9076 Acked Write 5 Done Sent Done false
29911 9077 Acked Write 10 Done Sent Done false
29912 9078 Acked Read 3 Done Sent Done false
29913 9079 Acked Write 10 Done Sent Done false
29914 9080 Acked Flush 0 Done Sent Done false
29915 9081 Acked Write 8 Done Sent Done false
29916 9082 Acked Read 4 Done Sent Done false
29917 9083 Acked Write 6 Done Sent Done false
29918 9084 Acked Write 10 Done Sent Done false
29919 9085 Acked Write 9 Done Sent Done false
29920 9086 Acked Read 9 Done Sent Done false
29921 9087 Acked Write 6 Done Sent Done false
29922 9088 Acked Read 10 Done Sent Done false
29923 9089 Acked Read 5 Done Sent Done false
29924 9090 Acked Write 2 Done Sent Done false
29925 9091 Acked Write 6 Done Sent Done false
29926 9092 Acked Write 1 Done Sent Done false
29927 9093 Acked Read 2 Done Sent Done false
29928 9094 Acked Read 7 Done Sent Done false
29929 9095 Acked Read 10 Done Sent Done false
29930 9096 Acked Flush 0 Done Sent Done false
29931 9097 Acked Read 2 Done Sent Done false
29932 9098 Acked Read 10 Done Sent Done false
29933 9099 Acked Write 6 Done Sent Done false
29934 9100 Acked Read 3 Done Sent Done false
29935 9101 Acked Read 7 Done Sent Done false
29936 9102 Acked Read 3 Done Sent Done false
29937 9103 Acked Read 2 Done Sent Done false
29938 9104 Acked Flush 0 Done Sent Done false
29939 9105 Acked Write 3 Done Sent Done false
29940 9106 Acked Read 10 Done Sent Done false
29941 9107 Acked Write 8 Done Sent Done false
29942 9108 Acked Write 10 Done Sent Done false
29943 9109 Acked Write 7 Done Sent Done false
29944 9110 Acked Write 7 Done Sent Done false
29945 9111 Acked Read 10 Done Sent Done false
29946 9112 Acked Read 6 Done Sent Done false
29947 9113 Acked Read 7 Done Sent Done false
29948 9114 Acked Read 7 Done Sent Done false
29949 9115 Acked Read 2 Done Sent Done false
29950 9116 Acked Read 10 Done Sent Done false
29951 9117 Acked Read 7 Done Sent Done false
29952 9118 Acked Write 8 Done Sent Done false
29953 9119 Acked Read 9 Done Sent Done false
29954 9120 Acked Read 4 Done Sent Done false
29955 9121 Acked Read 8 Done Sent Done false
29956 9122 Acked Read 5 Done Sent Done false
29957 9123 Acked Flush 0 Done Sent Done false
29958 9124 Acked Write 5 Done Sent Done false
29959 9125 Acked Write 4 Done Sent Done false
29960 9126 Acked Write 2 Done Sent Done false
29961 9127 Acked Read 10 Done Sent Done false
29962 9128 Acked Write 3 Done Sent Done false
29963 9129 Acked Write 9 Done Sent Done false
29964 9130 Acked Write 2 Done Sent Done false
29965 9131 Acked Read 7 Done Sent Done false
29966 9132 Acked Write 5 Done Sent Done false
29967 9133 Acked Write 9 Done Sent Done false
29968 9134 Acked Read 6 Done Sent Done false
29969 9135 Acked Read 8 Done Sent Done false
29970 9136 Acked Write 2 Done Sent Done false
29971 9137 Acked Write 3 Done Sent Done false
29972 9138 Acked Read 9 Done Sent Done false
29973 9139 Acked Write 1 Done Sent Done false
29974 9140 Acked Flush 0 Done Sent Done false
29975 9141 Acked Read 10 Done Sent Done false
29976 9142 Acked Write 9 Done Sent Done false
29977 9143 Acked Write 6 Done Sent Done false
29978 9144 Acked Read 5 Done Sent Done false
29979 9145 Acked Write 10 Done Sent Done false
29980 9146 Acked Write 1 Done Sent Done false
29981 9147 Acked Read 1 Done Sent Done false
29982 9148 Acked Write 8 Done Sent Done false
29983 9149 Acked Write 1 Done Sent Done false
29984 9150 Acked Read 1 Done Sent Done false
29985 9151 Acked Write 2 Done Sent Done false
29986 9152 Acked Write 7 Done Sent Done false
29987 9153 Acked Read 9 Done Sent Done false
29988 9154 Acked Write 8 Done Sent Done false
29989 9155 Acked Read 3 Done Sent Done false
29990 9156 Acked Flush 0 Done Sent Done false
29991 9157 Acked Flush 0 Done Sent Done false
29992 9158 Acked Write 3 Done Sent Done false
29993 9159 Acked Read 9 Done Sent Done false
29994 9160 Acked Read 6 Done Sent Done false
29995 9161 Acked Write 2 Done Sent Done false
29996 9162 Acked Write 8 Done Sent Done false
29997 9163 Acked Write 3 Done Sent Done false
29998 9164 Acked Write 6 Sent Sent Done false
29999 STATES DS:0 DS:1 DS:2 TOTAL
30000 Sent 1 302 0 303
30001 Done 301 0 302 603
30002 Skipped 0 0 0 0
30003 Error 0 0 0 0
30004 Last Flush: 9157 8862 9157
30005 Downstairs last five completed: 8862 8861 8860 8859 8858
30006 Upstairs last five completed: 9164 9163 9162 9161 9160
30007 CLIENT: Up:0 ds:302 act:2
30008 ----------------------------------------------------------------
30009 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
30010 States: Active Offline Active
30011 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30012 8863 Acked Write 6 Done Sent Done false
30013 8864 Acked Read 9 Done Sent Done false
30014 8865 Acked Read 6 Done Sent Done false
30015 8866 Acked Flush 0 Done Sent Done false
30016 8867 Acked Write 3 Done Sent Done false
30017 8868 Acked Flush 0 Done Sent Done false
30018 8869 Acked Read 6 Done Sent Done false
30019 8870 Acked Write 1 Done Sent Done false
30020 8871 Acked Write 10 Done Sent Done false
30021 8872 Acked Flush 0 Done Sent Done false
30022 8873 Acked Flush 0 Done Sent Done false
30023 8874 Acked Read 4 Done Sent Done false
30024 8875 Acked Read 10 Done Sent Done false
30025 8876 Acked Read 9 Done Sent Done false
30026 8877 Acked Read 2 Done Sent Done false
30027 8878 Acked Flush 0 Done Sent Done false
30028 8879 Acked Write 6 Done Sent Done false
30029 8880 Acked Read 7 Done Sent Done false
30030 8881 Acked Write 3 Done Sent Done false
30031 8882 Acked Write 10 Done Sent Done false
30032 8883 Acked Flush 0 Done Sent Done false
30033 8884 Acked Read 8 Done Sent Done false
30034 8885 Acked Read 10 Done Sent Done false
30035 8886 Acked Read 10 Done Sent Done false
30036 8887 Acked Write 4 Done Sent Done false
30037 8888 Acked Write 5 Done Sent Done false
30038 8889 Acked Read 9 Done Sent Done false
30039 8890 Acked Write 7 Done Sent Done false
30040 8891 Acked Read 2 Done Sent Done false
30041 8892 Acked Read 8 Done Sent Done false
30042 8893 Acked Write 6 Done Sent Done false
30043 8894 Acked Write 7 Done Sent Done false
30044 8895 Acked Flush 0 Done Sent Done false
30045 8896 Acked Flush 0 Done Sent Done false
30046 8897 Acked Flush 0 Done Sent Done false
30047 8898 Acked Read 1 Done Sent Done false
30048 8899 Acked Flush 0 Done Sent Done false
30049 8900 Acked Read 7 Done Sent Done false
30050 8901 Acked Write 9 Done Sent Done false
30051 8902 Acked Read 8 Done Sent Done false
30052 8903 Acked Flush 0 Done Sent Done false
30053 8904 Acked Read 3 Done Sent Done false
30054 8905 Acked Read 1 Done Sent Done false
30055 8906 Acked Write 3 Done Sent Done false
30056 8907 Acked Write 8 Done Sent Done false
30057 8908 Acked Read 9 Done Sent Done false
30058 8909 Acked Read 10 Done Sent Done false
30059 8910 Acked Read 4 Done Sent Done false
30060 8911 Acked Read 1 Done Sent Done false
30061 8912 Acked Read 7 Done Sent Done false
30062 8913 Acked Read 8 Done Sent Done false
30063 8914 Acked Read 9 Done Sent Done false
30064 8915 Acked Write 7 Done Sent Done false
30065 8916 Acked Read 5 Done Sent Done false
30066 8917 Acked Flush 0 Done Sent Done false
30067 8918 Acked Read 6 Done Sent Done false
30068 8919 Acked Flush 0 Done Sent Done false
30069 8920 Acked Read 10 Done Sent Done false
30070 8921 Acked Read 1 Done Sent Done false
30071 8922 Acked Read 6 Done Sent Done false
30072 8923 Acked Write 9 Done Sent Done false
30073 8924 Acked Write 7 Done Sent Done false
30074 8925 Acked Read 6 Done Sent Done false
30075 8926 Acked Read 8 Done Sent Done false
30076 8927 Acked Write 10 Done Sent Done false
30077 8928 Acked Read 2 Done Sent Done false
30078 8929 Acked Read 4 Done Sent Done false
30079 8930 Acked Write 7 Done Sent Done false
30080 8931 Acked Write 10 Done Sent Done false
30081 8932 Acked Read 1 Done Sent Done false
30082 8933 Acked Write 1 Done Sent Done false
30083 8934 Acked Read 7 Done Sent Done false
30084 8935 Acked Write 10 Done Sent Done false
30085 8936 Acked Read 9 Done Sent Done false
30086 8937 Acked Write 9 Done Sent Done false
30087 8938 Acked Write 10 Done Sent Done false
30088 8939 Acked Read 1 Done Sent Done false
30089 8940 Acked Read 1 Done Sent Done false
30090 8941 Acked Read 8 Done Sent Done false
30091 8942 Acked Write 10 Done Sent Done false
30092 8943 Acked Read 5 Done Sent Done false
30093 8944 Acked Read 7 Done Sent Done false
30094 8945 Acked Write 1 Done Sent Done false
30095 8946 Acked Read 1 Done Sent Done false
30096 8947 Acked Read 7 Done Sent Done false
30097 8948 Acked Write 1 Done Sent Done false
30098 8949 Acked Write 1 Done Sent Done false
30099 8950 Acked Flush 0 Done Sent Done false
30100 8951 Acked Flush 0 Done Sent Done false
30101 8952 Acked Read 4 Done Sent Done false
30102 8953 Acked Write 9 Done Sent Done false
30103 8954 Acked Read 9 Done Sent Done false
30104 8955 Acked Read 10 Done Sent Done false
30105 8956 Acked Write 4 Done Sent Done false
30106 8957 Acked Read 10 Done Sent Done false
30107 8958 Acked Write 4 Done Sent Done false
30108 8959 Acked Write 3 Done Sent Done false
30109 8960 Acked Write 10 Done Sent Done false
30110 8961 Acked Flush 0 Done Sent Done false
30111 8962 Acked Read 5 Done Sent Done false
30112 8963 Acked Write 5 Done Sent Done false
30113 8964 Acked Read 7 Done Sent Done false
30114 8965 Acked Flush 0 Done Sent Done false
30115 8966 Acked Write 6 Done Sent Done false
30116 8967 Acked Write 4 Done Sent Done false
30117 8968 Acked Write 7 Done Sent Done false
30118 8969 Acked Read 5 Done Sent Done false
30119 8970 Acked Read 4 Done Sent Done false
30120 8971 Acked Write 9 Done Sent Done false
30121 8972 Acked Write 6 Done Sent Done false
30122 8973 Acked Write 3 Done Sent Done false
30123 8974 Acked Write 5 Done Sent Done false
30124 8975 Acked Write 4 Done Sent Done false
30125 8976 Acked Write 1 Done Sent Done false
30126 8977 Acked Read 10 Done Sent Done false
30127 8978 Acked Flush 0 Done Sent Done false
30128 8979 Acked Read 2 Done Sent Done false
30129 8980 Acked Read 6 Done Sent Done false
30130 8981 Acked Read 4 Done Sent Done false
30131 8982 Acked Write 1 Done Sent Done false
30132 8983 Acked Write 6 Done Sent Done false
30133 8984 Acked Read 7 Done Sent Done false
30134 8985 Acked Read 6 Done Sent Done false
30135 8986 Acked Write 5 Done Sent Done false
30136 8987 Acked Read 5 Done Sent Done false
30137 8988 Acked Read 7 Done Sent Done false
30138 8989 Acked Flush 0 Done Sent Done false
30139 8990 Acked Read 4 Done Sent Done false
30140 8991 Acked Read 9 Done Sent Done false
30141 8992 Acked Write 8 Done Sent Done false
30142 8993 Acked Flush 0 Done Sent Done false
30143 8994 Acked Read 4 Done Sent Done false
30144 8995 Acked Read 3 Done Sent Done false
30145 8996 Acked Read 8 Done Sent Done false
30146 8997 Acked Read 5 Done Sent Done false
30147 8998 Acked Write 7 Done Sent Done false
30148 8999 Acked Read 9 Done Sent Done false
30149 9000 Acked Write 2 Done Sent Done false
30150 9001 Acked Read 1 Done Sent Done false
30151 9002 Acked Write 9 Done Sent Done false
30152 9003 Acked Write 6 Done Sent Done false
30153 9004 Acked Write 4 Done Sent Done false
30154 9005 Acked Read 9 Done Sent Done false
30155 9006 Acked Write 3 Done Sent Done false
30156 9007 Acked Read 8 Done Sent Done false
30157 9008 Acked Write 1 Done Sent Done false
30158 9009 Acked Read 10 Done Sent Done false
30159 9010 Acked Write 8 Done Sent Done false
30160 9011 Acked Read 6 Done Sent Done false
30161 9012 Acked Write 9 Done Sent Done false
30162 9013 Acked Read 8 Done Sent Done false
30163 9014 Acked Flush 0 Done Sent Done false
30164 9015 Acked Write 10 Done Sent Done false
30165 9016 Acked Write 1 Done Sent Done false
30166 9017 Acked Read 9 Done Sent Done false
30167 9018 Acked Write 3 Done Sent Done false
30168 9019 Acked Read 7 Done Sent Done false
30169 9020 Acked Read 7 Done Sent Done false
30170 9021 Acked Write 10 Done Sent Done false
30171 9022 Acked Write 4 Done Sent Done false
30172 9023 Acked Write 4 Done Sent Done false
30173 9024 Acked Write 2 Done Sent Done false
30174 9025 Acked Flush 0 Done Sent Done false
30175 9026 Acked Write 3 Done Sent Done false
30176 9027 Acked Read 5 Done Sent Done false
30177 9028 Acked Read 8 Done Sent Done false
30178 9029 Acked Write 8 Done Sent Done false
30179 9030 Acked Read 6 Done Sent Done false
30180 9031 Acked Flush 0 Done Sent Done false
30181 9032 Acked Read 7 Done Sent Done false
30182 9033 Acked Read 1 Done Sent Done false
30183 9034 Acked Write 10 Done Sent Done false
30184 9035 Acked Read 1 Done Sent Done false
30185 9036 Acked Read 5 Done Sent Done false
30186 9037 Acked Write 5 Done Sent Done false
30187 9038 Acked Write 1 Done Sent Done false
30188 9039 Acked Write 6 Done Sent Done false
30189 9040 Acked Write 10 Done Sent Done false
30190 9041 Acked Write 8 Done Sent Done false
30191 9042 Acked Read 9 Done Sent Done false
30192 9043 Acked Read 5 Done Sent Done false
30193 9044 Acked Read 1 Done Sent Done false
30194 9045 Acked Read 6 Done Sent Done false
30195 9046 Acked Flush 0 Done Sent Done false
30196 9047 Acked Write 4 Done Sent Done false
30197 9048 Acked Write 7 Done Sent Done false
30198 9049 Acked Flush 0 Done Sent Done false
30199 9050 Acked Write 2 Done Sent Done false
30200 9051 Acked Write 2 Done Sent Done false
30201 9052 Acked Write 1 Done Sent Done false
30202 9053 Acked Write 9 Done Sent Done false
30203 9054 Acked Write 5 Done Sent Done false
30204 9055 Acked Read 9 Done Sent Done false
30205 9056 Acked Write 9 Done Sent Done false
30206 9057 Acked Read 8 Done Sent Done false
30207 9058 Acked Read 10 Done Sent Done false
30208 9059 Acked Read 8 Done Sent Done false
30209 9060 Acked Read 3 Done Sent Done false
30210 9061 Acked Write 6 Done Sent Done false
30211 9062 Acked Write 5 Done Sent Done false
30212 9063 Acked Write 3 Done Sent Done false
30213 9064 Acked Read 1 Done Sent Done false
30214 9065 Acked Read 2 Done Sent Done false
30215 9066 Acked Write 4 Done Sent Done false
30216 9067 Acked Read 10 Done Sent Done false
30217 9068 Acked Read 8 Done Sent Done false
30218 9069 Acked Read 4 Done Sent Done false
30219 9070 Acked Write 8 Done Sent Done false
30220 9071 Acked Write 7 Done Sent Done false
30221 9072 Acked Write 10 Done Sent Done false
30222 9073 Acked Write 2 Done Sent Done false
30223 9074 Acked Read 5 Done Sent Done false
30224 9075 Acked Flush 0 Done Sent Done false
30225 9076 Acked Write 5 Done Sent Done false
30226 9077 Acked Write 10 Done Sent Done false
30227 9078 Acked Read 3 Done Sent Done false
30228 9079 Acked Write 10 Done Sent Done false
30229 9080 Acked Flush 0 Done Sent Done false
30230 9081 Acked Write 8 Done Sent Done false
30231 9082 Acked Read 4 Done Sent Done false
30232 9083 Acked Write 6 Done Sent Done false
30233 9084 Acked Write 10 Done Sent Done false
30234 9085 Acked Write 9 Done Sent Done false
30235 9086 Acked Read 9 Done Sent Done false
30236 9087 Acked Write 6 Done Sent Done false
30237 9088 Acked Read 10 Done Sent Done false
30238 9089 Acked Read 5 Done Sent Done false
30239 9090 Acked Write 2 Done Sent Done false
30240 9091 Acked Write 6 Done Sent Done false
30241 9092 Acked Write 1 Done Sent Done false
30242 9093 Acked Read 2 Done Sent Done false
30243 9094 Acked Read 7 Done Sent Done false
30244 9095 Acked Read 10 Done Sent Done false
30245 9096 Acked Flush 0 Done Sent Done false
30246 9097 Acked Read 2 Done Sent Done false
30247 9098 Acked Read 10 Done Sent Done false
30248 9099 Acked Write 6 Done Sent Done false
30249 9100 Acked Read 3 Done Sent Done false
30250 9101 Acked Read 7 Done Sent Done false
30251 9102 Acked Read 3 Done Sent Done false
30252 9103 Acked Read 2 Done Sent Done false
30253 9104 Acked Flush 0 Done Sent Done false
30254 9105 Acked Write 3 Done Sent Done false
30255 9106 Acked Read 10 Done Sent Done false
30256 9107 Acked Write 8 Done Sent Done false
30257 9108 Acked Write 10 Done Sent Done false
30258 9109 Acked Write 7 Done Sent Done false
30259 9110 Acked Write 7 Done Sent Done false
30260 9111 Acked Read 10 Done Sent Done false
30261 9112 Acked Read 6 Done Sent Done false
30262 9113 Acked Read 7 Done Sent Done false
30263 9114 Acked Read 7 Done Sent Done false
30264 9115 Acked Read 2 Done Sent Done false
30265 9116 Acked Read 10 Done Sent Done false
30266 9117 Acked Read 7 Done Sent Done false
30267 9118 Acked Write 8 Done Sent Done false
30268 9119 Acked Read 9 Done Sent Done false
30269 9120 Acked Read 4 Done Sent Done false
30270 9121 Acked Read 8 Done Sent Done false
30271 9122 Acked Read 5 Done Sent Done false
30272 9123 Acked Flush 0 Done Sent Done false
30273 9124 Acked Write 5 Done Sent Done false
30274 9125 Acked Write 4 Done Sent Done false
30275 9126 Acked Write 2 Done Sent Done false
30276 9127 Acked Read 10 Done Sent Done false
30277 9128 Acked Write 3 Done Sent Done false
30278 9129 Acked Write 9 Done Sent Done false
30279 9130 Acked Write 2 Done Sent Done false
30280 9131 Acked Read 7 Done Sent Done false
30281 9132 Acked Write 5 Done Sent Done false
30282 9133 Acked Write 9 Done Sent Done false
30283 9134 Acked Read 6 Done Sent Done false
30284 9135 Acked Read 8 Done Sent Done false
30285 9136 Acked Write 2 Done Sent Done false
30286 9137 Acked Write 3 Done Sent Done false
30287 9138 Acked Read 9 Done Sent Done false
30288 9139 Acked Write 1 Done Sent Done false
30289 9140 Acked Flush 0 Done Sent Done false
30290 9141 Acked Read 10 Done Sent Done false
30291 9142 Acked Write 9 Done Sent Done false
30292 9143 Acked Write 6 Done Sent Done false
30293 9144 Acked Read 5 Done Sent Done false
30294 9145 Acked Write 10 Done Sent Done false
30295 9146 Acked Write 1 Done Sent Done false
30296 9147 Acked Read 1 Done Sent Done false
30297 9148 Acked Write 8 Done Sent Done false
30298 9149 Acked Write 1 Done Sent Done false
30299 9150 Acked Read 1 Done Sent Done false
30300 9151 Acked Write 2 Done Sent Done false
30301 9152 Acked Write 7 Done Sent Done false
30302 9153 Acked Read 9 Done Sent Done false
30303 9154 Acked Write 8 Done Sent Done false
30304 9155 Acked Read 3 Done Sent Done false
30305 9156 Acked Flush 0 Done Sent Done false
30306 9157 Acked Flush 0 Done Sent Done false
30307 9158 Acked Write 3 Done Sent Done false
30308 9159 Acked Read 9 Done Sent Done false
30309 9160 Acked Read 6 Done Sent Done false
30310 9161 Acked Write 2 Done Sent Done false
30311 9162 Acked Write 8 Done Sent Done false
30312 9163 Acked Write 3 Done Sent Done false
30313 9164 Acked Write 6 Done Sent Done false
30314 9165 Acked Flush 0 Done Sent Done false
30315 STATES DS:0 DS:1 DS:2 TOTAL
30316 Sent 0 303 0 303
30317 Done 303 0 303 606
30318 Skipped 0 0 0 0
30319 Error 0 0 0 0
30320 Last Flush: 9165 8862 9165
30321 Downstairs last five completed: 8862 8861 8860 8859 8858
30322 Upstairs last five completed: 9165 9164 9163 9162 9161
30323 CLIENT: Up:0 ds:303 act:2
303242024-11-07T14:09:39.497ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
303252024-11-07T14:09:39.497ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
303262024-11-07T14:09:39.497ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
303272024-11-07T14:09:39.497ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
303282024-11-07T14:09:39.497ZINFOcrucible: client task is exiting = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
30329 [8820] got start action
30330 [8820] I am going to restart
30331 Starting downstairs at port 8820
30332 Make output file at "/tmp/dsc/downstairs-8820.txt"
30333 [8820][1] reports Starting
30334 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1226)
30335 [8820][1] reports Running
30336 ----------------------------------------------------------------
30337 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
30338 States: Active Offline Active
30339 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30340 8863 Acked Write 6 Done Sent Done false
30341 8864 Acked Read 9 Done Sent Done false
30342 8865 Acked Read 6 Done Sent Done false
30343 8866 Acked Flush 0 Done Sent Done false
30344 8867 Acked Write 3 Done Sent Done false
30345 8868 Acked Flush 0 Done Sent Done false
30346 8869 Acked Read 6 Done Sent Done false
30347 8870 Acked Write 1 Done Sent Done false
30348 8871 Acked Write 10 Done Sent Done false
30349 8872 Acked Flush 0 Done Sent Done false
30350 8873 Acked Flush 0 Done Sent Done false
30351 8874 Acked Read 4 Done Sent Done false
30352 8875 Acked Read 10 Done Sent Done false
30353 8876 Acked Read 9 Done Sent Done false
30354 8877 Acked Read 2 Done Sent Done false
30355 8878 Acked Flush 0 Done Sent Done false
30356 8879 Acked Write 6 Done Sent Done false
30357 8880 Acked Read 7 Done Sent Done false
30358 8881 Acked Write 3 Done Sent Done false
30359 8882 Acked Write 10 Done Sent Done false
30360 8883 Acked Flush 0 Done Sent Done false
30361 8884 Acked Read 8 Done Sent Done false
30362 8885 Acked Read 10 Done Sent Done false
30363 8886 Acked Read 10 Done Sent Done false
30364 8887 Acked Write 4 Done Sent Done false
30365 8888 Acked Write 5 Done Sent Done false
30366 8889 Acked Read 9 Done Sent Done false
30367 8890 Acked Write 7 Done Sent Done false
30368 8891 Acked Read 2 Done Sent Done false
30369 8892 Acked Read 8 Done Sent Done false
30370 8893 Acked Write 6 Done Sent Done false
30371 8894 Acked Write 7 Done Sent Done false
30372 8895 Acked Flush 0 Done Sent Done false
30373 8896 Acked Flush 0 Done Sent Done false
30374 8897 Acked Flush 0 Done Sent Done false
30375 8898 Acked Read 1 Done Sent Done false
30376 8899 Acked Flush 0 Done Sent Done false
30377 8900 Acked Read 7 Done Sent Done false
30378 8901 Acked Write 9 Done Sent Done false
30379 8902 Acked Read 8 Done Sent Done false
30380 8903 Acked Flush 0 Done Sent Done false
30381 8904 Acked Read 3 Done Sent Done false
30382 8905 Acked Read 1 Done Sent Done false
30383 8906 Acked Write 3 Done Sent Done false
30384 8907 Acked Write 8 Done Sent Done false
30385 8908 Acked Read 9 Done Sent Done false
30386 8909 Acked Read 10 Done Sent Done false
30387 8910 Acked Read 4 Done Sent Done false
30388 8911 Acked Read 1 Done Sent Done false
30389 8912 Acked Read 7 Done Sent Done false
30390 8913 Acked Read 8 Done Sent Done false
30391 8914 Acked Read 9 Done Sent Done false
30392 8915 Acked Write 7 Done Sent Done false
30393 8916 Acked Read 5 Done Sent Done false
30394 8917 Acked Flush 0 Done Sent Done false
30395 8918 Acked Read 6 Done Sent Done false
30396 8919 Acked Flush 0 Done Sent Done false
30397 8920 Acked Read 10 Done Sent Done false
30398 8921 Acked Read 1 Done Sent Done false
30399 8922 Acked Read 6 Done Sent Done false
30400 8923 Acked Write 9 Done Sent Done false
30401 8924 Acked Write 7 Done Sent Done false
30402 8925 Acked Read 6 Done Sent Done false
30403 8926 Acked Read 8 Done Sent Done false
30404 8927 Acked Write 10 Done Sent Done false
30405 8928 Acked Read 2 Done Sent Done false
30406 8929 Acked Read 4 Done Sent Done false
30407 8930 Acked Write 7 Done Sent Done false
30408 8931 Acked Write 10 Done Sent Done false
30409 8932 Acked Read 1 Done Sent Done false
30410 8933 Acked Write 1 Done Sent Done false
30411 8934 Acked Read 7 Done Sent Done false
30412 8935 Acked Write 10 Done Sent Done false
30413 8936 Acked Read 9 Done Sent Done false
30414 8937 Acked Write 9 Done Sent Done false
30415 8938 Acked Write 10 Done Sent Done false
30416 8939 Acked Read 1 Done Sent Done false
30417 8940 Acked Read 1 Done Sent Done false
30418 8941 Acked Read 8 Done Sent Done false
30419 8942 Acked Write 10 Done Sent Done false
30420 8943 Acked Read 5 Done Sent Done false
30421 8944 Acked Read 7 Done Sent Done false
30422 8945 Acked Write 1 Done Sent Done false
30423 8946 Acked Read 1 Done Sent Done false
30424 8947 Acked Read 7 Done Sent Done false
30425 8948 Acked Write 1 Done Sent Done false
30426 8949 Acked Write 1 Done Sent Done false
30427 8950 Acked Flush 0 Done Sent Done false
30428 8951 Acked Flush 0 Done Sent Done false
30429 8952 Acked Read 4 Done Sent Done false
30430 8953 Acked Write 9 Done Sent Done false
30431 8954 Acked Read 9 Done Sent Done false
30432 8955 Acked Read 10 Done Sent Done false
30433 8956 Acked Write 4 Done Sent Done false
30434 8957 Acked Read 10 Done Sent Done false
30435 8958 Acked Write 4 Done Sent Done false
30436 8959 Acked Write 3 Done Sent Done false
30437 8960 Acked Write 10 Done Sent Done false
30438 8961 Acked Flush 0 Done Sent Done false
30439 8962 Acked Read 5 Done Sent Done false
30440 8963 Acked Write 5 Done Sent Done false
30441 8964 Acked Read 7 Done Sent Done false
30442 8965 Acked Flush 0 Done Sent Done false
30443 8966 Acked Write 6 Done Sent Done false
30444 8967 Acked Write 4 Done Sent Done false
30445 8968 Acked Write 7 Done Sent Done false
30446 8969 Acked Read 5 Done Sent Done false
30447 8970 Acked Read 4 Done Sent Done false
30448 8971 Acked Write 9 Done Sent Done false
30449 8972 Acked Write 6 Done Sent Done false
30450 8973 Acked Write 3 Done Sent Done false
30451 8974 Acked Write 5 Done Sent Done false
30452 8975 Acked Write 4 Done Sent Done false
30453 8976 Acked Write 1 Done Sent Done false
30454 8977 Acked Read 10 Done Sent Done false
30455 8978 Acked Flush 0 Done Sent Done false
30456 8979 Acked Read 2 Done Sent Done false
30457 8980 Acked Read 6 Done Sent Done false
30458 8981 Acked Read 4 Done Sent Done false
30459 8982 Acked Write 1 Done Sent Done false
30460 8983 Acked Write 6 Done Sent Done false
30461 8984 Acked Read 7 Done Sent Done false
30462 8985 Acked Read 6 Done Sent Done false
30463 8986 Acked Write 5 Done Sent Done false
30464 8987 Acked Read 5 Done Sent Done false
30465 8988 Acked Read 7 Done Sent Done false
30466 8989 Acked Flush 0 Done Sent Done false
30467 8990 Acked Read 4 Done Sent Done false
30468 8991 Acked Read 9 Done Sent Done false
30469 8992 Acked Write 8 Done Sent Done false
30470 8993 Acked Flush 0 Done Sent Done false
30471 8994 Acked Read 4 Done Sent Done false
30472 8995 Acked Read 3 Done Sent Done false
30473 8996 Acked Read 8 Done Sent Done false
30474 8997 Acked Read 5 Done Sent Done false
30475 8998 Acked Write 7 Done Sent Done false
30476 8999 Acked Read 9 Done Sent Done false
30477 9000 Acked Write 2 Done Sent Done false
30478 9001 Acked Read 1 Done Sent Done false
30479 9002 Acked Write 9 Done Sent Done false
30480 9003 Acked Write 6 Done Sent Done false
30481 9004 Acked Write 4 Done Sent Done false
30482 9005 Acked Read 9 Done Sent Done false
30483 9006 Acked Write 3 Done Sent Done false
30484 9007 Acked Read 8 Done Sent Done false
30485 9008 Acked Write 1 Done Sent Done false
30486 9009 Acked Read 10 Done Sent Done false
30487 9010 Acked Write 8 Done Sent Done false
30488 9011 Acked Read 6 Done Sent Done false
30489 9012 Acked Write 9 Done Sent Done false
30490 9013 Acked Read 8 Done Sent Done false
30491 9014 Acked Flush 0 Done Sent Done false
30492 9015 Acked Write 10 Done Sent Done false
30493 9016 Acked Write 1 Done Sent Done false
30494 9017 Acked Read 9 Done Sent Done false
30495 9018 Acked Write 3 Done Sent Done false
30496 9019 Acked Read 7 Done Sent Done false
30497 9020 Acked Read 7 Done Sent Done false
30498 9021 Acked Write 10 Done Sent Done false
30499 9022 Acked Write 4 Done Sent Done false
30500 9023 Acked Write 4 Done Sent Done false
30501 9024 Acked Write 2 Done Sent Done false
30502 9025 Acked Flush 0 Done Sent Done false
30503 9026 Acked Write 3 Done Sent Done false
30504 9027 Acked Read 5 Done Sent Done false
30505 9028 Acked Read 8 Done Sent Done false
30506 9029 Acked Write 8 Done Sent Done false
30507 9030 Acked Read 6 Done Sent Done false
30508 9031 Acked Flush 0 Done Sent Done false
30509 9032 Acked Read 7 Done Sent Done false
30510 9033 Acked Read 1 Done Sent Done false
30511 9034 Acked Write 10 Done Sent Done false
30512 9035 Acked Read 1 Done Sent Done false
30513 9036 Acked Read 5 Done Sent Done false
30514 9037 Acked Write 5 Done Sent Done false
30515 9038 Acked Write 1 Done Sent Done false
30516 9039 Acked Write 6 Done Sent Done false
30517 9040 Acked Write 10 Done Sent Done false
30518 9041 Acked Write 8 Done Sent Done false
30519 9042 Acked Read 9 Done Sent Done false
30520 9043 Acked Read 5 Done Sent Done false
30521 9044 Acked Read 1 Done Sent Done false
30522 9045 Acked Read 6 Done Sent Done false
30523 9046 Acked Flush 0 Done Sent Done false
30524 9047 Acked Write 4 Done Sent Done false
30525 9048 Acked Write 7 Done Sent Done false
30526 9049 Acked Flush 0 Done Sent Done false
30527 9050 Acked Write 2 Done Sent Done false
30528 9051 Acked Write 2 Done Sent Done false
30529 9052 Acked Write 1 Done Sent Done false
30530 9053 Acked Write 9 Done Sent Done false
30531 9054 Acked Write 5 Done Sent Done false
30532 9055 Acked Read 9 Done Sent Done false
30533 9056 Acked Write 9 Done Sent Done false
30534 9057 Acked Read 8 Done Sent Done false
30535 9058 Acked Read 10 Done Sent Done false
30536 9059 Acked Read 8 Done Sent Done false
30537 9060 Acked Read 3 Done Sent Done false
30538 9061 Acked Write 6 Done Sent Done false
30539 9062 Acked Write 5 Done Sent Done false
30540 9063 Acked Write 3 Done Sent Done false
30541 9064 Acked Read 1 Done Sent Done false
30542 9065 Acked Read 2 Done Sent Done false
30543 9066 Acked Write 4 Done Sent Done false
30544 9067 Acked Read 10 Done Sent Done false
30545 9068 Acked Read 8 Done Sent Done false
30546 9069 Acked Read 4 Done Sent Done false
30547 9070 Acked Write 8 Done Sent Done false
30548 9071 Acked Write 7 Done Sent Done false
30549 9072 Acked Write 10 Done Sent Done false
30550 9073 Acked Write 2 Done Sent Done false
30551 9074 Acked Read 5 Done Sent Done false
30552 9075 Acked Flush 0 Done Sent Done false
30553 9076 Acked Write 5 Done Sent Done false
30554 9077 Acked Write 10 Done Sent Done false
30555 9078 Acked Read 3 Done Sent Done false
30556 9079 Acked Write 10 Done Sent Done false
30557 9080 Acked Flush 0 Done Sent Done false
30558 9081 Acked Write 8 Done Sent Done false
30559 9082 Acked Read 4 Done Sent Done false
30560 9083 Acked Write 6 Done Sent Done false
30561 9084 Acked Write 10 Done Sent Done false
30562 9085 Acked Write 9 Done Sent Done false
30563 9086 Acked Read 9 Done Sent Done false
30564 9087 Acked Write 6 Done Sent Done false
30565 9088 Acked Read 10 Done Sent Done false
30566 9089 Acked Read 5 Done Sent Done false
30567 9090 Acked Write 2 Done Sent Done false
30568 9091 Acked Write 6 Done Sent Done false
30569 9092 Acked Write 1 Done Sent Done false
30570 9093 Acked Read 2 Done Sent Done false
30571 9094 Acked Read 7 Done Sent Done false
30572 9095 Acked Read 10 Done Sent Done false
30573 9096 Acked Flush 0 Done Sent Done false
30574 9097 Acked Read 2 Done Sent Done false
30575 9098 Acked Read 10 Done Sent Done false
30576 9099 Acked Write 6 Done Sent Done false
30577 9100 Acked Read 3 Done Sent Done false
30578 9101 Acked Read 7 Done Sent Done false
30579 9102 Acked Read 3 Done Sent Done false
30580 9103 Acked Read 2 Done Sent Done false
30581 9104 Acked Flush 0 Done Sent Done false
30582 9105 Acked Write 3 Done Sent Done false
30583 9106 Acked Read 10 Done Sent Done false
30584 9107 Acked Write 8 Done Sent Done false
30585 9108 Acked Write 10 Done Sent Done false
30586 9109 Acked Write 7 Done Sent Done false
30587 9110 Acked Write 7 Done Sent Done false
30588 9111 Acked Read 10 Done Sent Done false
30589 9112 Acked Read 6 Done Sent Done false
30590 9113 Acked Read 7 Done Sent Done false
30591 9114 Acked Read 7 Done Sent Done false
30592 9115 Acked Read 2 Done Sent Done false
30593 9116 Acked Read 10 Done Sent Done false
30594 9117 Acked Read 7 Done Sent Done false
30595 9118 Acked Write 8 Done Sent Done false
30596 9119 Acked Read 9 Done Sent Done false
30597 9120 Acked Read 4 Done Sent Done false
30598 9121 Acked Read 8 Done Sent Done false
30599 9122 Acked Read 5 Done Sent Done false
30600 9123 Acked Flush 0 Done Sent Done false
30601 9124 Acked Write 5 Done Sent Done false
30602 9125 Acked Write 4 Done Sent Done false
30603 9126 Acked Write 2 Done Sent Done false
30604 9127 Acked Read 10 Done Sent Done false
30605 9128 Acked Write 3 Done Sent Done false
30606 9129 Acked Write 9 Done Sent Done false
30607 9130 Acked Write 2 Done Sent Done false
30608 9131 Acked Read 7 Done Sent Done false
30609 9132 Acked Write 5 Done Sent Done false
30610 9133 Acked Write 9 Done Sent Done false
30611 9134 Acked Read 6 Done Sent Done false
30612 9135 Acked Read 8 Done Sent Done false
30613 9136 Acked Write 2 Done Sent Done false
30614 9137 Acked Write 3 Done Sent Done false
30615 9138 Acked Read 9 Done Sent Done false
30616 9139 Acked Write 1 Done Sent Done false
30617 9140 Acked Flush 0 Done Sent Done false
30618 9141 Acked Read 10 Done Sent Done false
30619 9142 Acked Write 9 Done Sent Done false
30620 9143 Acked Write 6 Done Sent Done false
30621 9144 Acked Read 5 Done Sent Done false
30622 9145 Acked Write 10 Done Sent Done false
30623 9146 Acked Write 1 Done Sent Done false
30624 9147 Acked Read 1 Done Sent Done false
30625 9148 Acked Write 8 Done Sent Done false
30626 9149 Acked Write 1 Done Sent Done false
30627 9150 Acked Read 1 Done Sent Done false
30628 9151 Acked Write 2 Done Sent Done false
30629 9152 Acked Write 7 Done Sent Done false
30630 9153 Acked Read 9 Done Sent Done false
30631 9154 Acked Write 8 Done Sent Done false
30632 9155 Acked Read 3 Done Sent Done false
30633 9156 Acked Flush 0 Done Sent Done false
30634 9157 Acked Flush 0 Done Sent Done false
30635 9158 Acked Write 3 Done Sent Done false
30636 9159 Acked Read 9 Done Sent Done false
30637 9160 Acked Read 6 Done Sent Done false
30638 9161 Acked Write 2 Done Sent Done false
30639 9162 Acked Write 8 Done Sent Done false
30640 9163 Acked Write 3 Done Sent Done false
30641 9164 Acked Write 6 Done Sent Done false
30642 9165 Acked Flush 0 Done Sent Done false
30643 STATES DS:0 DS:1 DS:2 TOTAL
30644 Sent 0 303 0 303
30645 Done 303 0 303 606
30646 Skipped 0 0 0 0
30647 Error 0 0 0 0
30648 Last Flush: 9165 8862 9165
30649 Downstairs last five completed: 8862 8861 8860 8859 8858
30650 Upstairs last five completed: 9165 9164 9163 9162 9161
30651 CLIENT: Up:0 ds:303 act:2
30652 ----------------------------------------------------------------
30653 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
30654 States: Active Offline Active
30655 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30656 8863 Acked Write 6 Done Sent Done false
30657 8864 Acked Read 9 Done Sent Done false
30658 8865 Acked Read 6 Done Sent Done false
30659 8866 Acked Flush 0 Done Sent Done false
30660 8867 Acked Write 3 Done Sent Done false
30661 8868 Acked Flush 0 Done Sent Done false
30662 8869 Acked Read 6 Done Sent Done false
30663 8870 Acked Write 1 Done Sent Done false
30664 8871 Acked Write 10 Done Sent Done false
30665 8872 Acked Flush 0 Done Sent Done false
30666 8873 Acked Flush 0 Done Sent Done false
30667 8874 Acked Read 4 Done Sent Done false
30668 8875 Acked Read 10 Done Sent Done false
30669 8876 Acked Read 9 Done Sent Done false
30670 8877 Acked Read 2 Done Sent Done false
30671 8878 Acked Flush 0 Done Sent Done false
30672 8879 Acked Write 6 Done Sent Done false
30673 8880 Acked Read 7 Done Sent Done false
30674 8881 Acked Write 3 Done Sent Done false
30675 8882 Acked Write 10 Done Sent Done false
30676 8883 Acked Flush 0 Done Sent Done false
30677 8884 Acked Read 8 Done Sent Done false
30678 8885 Acked Read 10 Done Sent Done false
30679 8886 Acked Read 10 Done Sent Done false
30680 8887 Acked Write 4 Done Sent Done false
30681 8888 Acked Write 5 Done Sent Done false
30682 8889 Acked Read 9 Done Sent Done false
30683 8890 Acked Write 7 Done Sent Done false
30684 8891 Acked Read 2 Done Sent Done false
30685 8892 Acked Read 8 Done Sent Done false
30686 8893 Acked Write 6 Done Sent Done false
30687 8894 Acked Write 7 Done Sent Done false
30688 8895 Acked Flush 0 Done Sent Done false
30689 8896 Acked Flush 0 Done Sent Done false
30690 8897 Acked Flush 0 Done Sent Done false
30691 8898 Acked Read 1 Done Sent Done false
30692 8899 Acked Flush 0 Done Sent Done false
30693 8900 Acked Read 7 Done Sent Done false
30694 8901 Acked Write 9 Done Sent Done false
30695 8902 Acked Read 8 Done Sent Done false
30696 8903 Acked Flush 0 Done Sent Done false
30697 8904 Acked Read 3 Done Sent Done false
30698 8905 Acked Read 1 Done Sent Done false
30699 8906 Acked Write 3 Done Sent Done false
30700 8907 Acked Write 8 Done Sent Done false
30701 8908 Acked Read 9 Done Sent Done false
30702 8909 Acked Read 10 Done Sent Done false
30703 8910 Acked Read 4 Done Sent Done false
30704 8911 Acked Read 1 Done Sent Done false
30705 8912 Acked Read 7 Done Sent Done false
30706 8913 Acked Read 8 Done Sent Done false
30707 8914 Acked Read 9 Done Sent Done false
30708 8915 Acked Write 7 Done Sent Done false
30709 8916 Acked Read 5 Done Sent Done false
30710 8917 Acked Flush 0 Done Sent Done false
30711 8918 Acked Read 6 Done Sent Done false
30712 8919 Acked Flush 0 Done Sent Done false
30713 8920 Acked Read 10 Done Sent Done false
30714 8921 Acked Read 1 Done Sent Done false
30715 8922 Acked Read 6 Done Sent Done false
30716 8923 Acked Write 9 Done Sent Done false
30717 8924 Acked Write 7 Done Sent Done false
30718 8925 Acked Read 6 Done Sent Done false
30719 8926 Acked Read 8 Done Sent Done false
30720 8927 Acked Write 10 Done Sent Done false
30721 8928 Acked Read 2 Done Sent Done false
30722 8929 Acked Read 4 Done Sent Done false
30723 8930 Acked Write 7 Done Sent Done false
30724 8931 Acked Write 10 Done Sent Done false
30725 8932 Acked Read 1 Done Sent Done false
30726 8933 Acked Write 1 Done Sent Done false
30727 8934 Acked Read 7 Done Sent Done false
30728 8935 Acked Write 10 Done Sent Done false
30729 8936 Acked Read 9 Done Sent Done false
30730 8937 Acked Write 9 Done Sent Done false
30731 8938 Acked Write 10 Done Sent Done false
30732 8939 Acked Read 1 Done Sent Done false
30733 8940 Acked Read 1 Done Sent Done false
30734 8941 Acked Read 8 Done Sent Done false
30735 8942 Acked Write 10 Done Sent Done false
30736 8943 Acked Read 5 Done Sent Done false
30737 8944 Acked Read 7 Done Sent Done false
30738 8945 Acked Write 1 Done Sent Done false
30739 8946 Acked Read 1 Done Sent Done false
30740 8947 Acked Read 7 Done Sent Done false
30741 8948 Acked Write 1 Done Sent Done false
30742 8949 Acked Write 1 Done Sent Done false
30743 8950 Acked Flush 0 Done Sent Done false
30744 8951 Acked Flush 0 Done Sent Done false
30745 8952 Acked Read 4 Done Sent Done false
30746 8953 Acked Write 9 Done Sent Done false
30747 8954 Acked Read 9 Done Sent Done false
30748 8955 Acked Read 10 Done Sent Done false
30749 8956 Acked Write 4 Done Sent Done false
30750 8957 Acked Read 10 Done Sent Done false
30751 8958 Acked Write 4 Done Sent Done false
30752 8959 Acked Write 3 Done Sent Done false
30753 8960 Acked Write 10 Done Sent Done false
30754 8961 Acked Flush 0 Done Sent Done false
30755 8962 Acked Read 5 Done Sent Done false
30756 8963 Acked Write 5 Done Sent Done false
30757 8964 Acked Read 7 Done Sent Done false
30758 8965 Acked Flush 0 Done Sent Done false
30759 8966 Acked Write 6 Done Sent Done false
30760 8967 Acked Write 4 Done Sent Done false
30761 8968 Acked Write 7 Done Sent Done false
30762 8969 Acked Read 5 Done Sent Done false
30763 8970 Acked Read 4 Done Sent Done false
30764 8971 Acked Write 9 Done Sent Done false
30765 8972 Acked Write 6 Done Sent Done false
30766 8973 Acked Write 3 Done Sent Done false
30767 8974 Acked Write 5 Done Sent Done false
30768 8975 Acked Write 4 Done Sent Done false
30769 8976 Acked Write 1 Done Sent Done false
30770 8977 Acked Read 10 Done Sent Done false
30771 8978 Acked Flush 0 Done Sent Done false
30772 8979 Acked Read 2 Done Sent Done false
30773 8980 Acked Read 6 Done Sent Done false
30774 8981 Acked Read 4 Done Sent Done false
30775 8982 Acked Write 1 Done Sent Done false
30776 8983 Acked Write 6 Done Sent Done false
30777 8984 Acked Read 7 Done Sent Done false
30778 8985 Acked Read 6 Done Sent Done false
30779 8986 Acked Write 5 Done Sent Done false
30780 8987 Acked Read 5 Done Sent Done false
30781 8988 Acked Read 7 Done Sent Done false
30782 8989 Acked Flush 0 Done Sent Done false
30783 8990 Acked Read 4 Done Sent Done false
30784 8991 Acked Read 9 Done Sent Done false
30785 8992 Acked Write 8 Done Sent Done false
30786 8993 Acked Flush 0 Done Sent Done false
30787 8994 Acked Read 4 Done Sent Done false
30788 8995 Acked Read 3 Done Sent Done false
30789 8996 Acked Read 8 Done Sent Done false
30790 8997 Acked Read 5 Done Sent Done false
30791 8998 Acked Write 7 Done Sent Done false
30792 8999 Acked Read 9 Done Sent Done false
30793 9000 Acked Write 2 Done Sent Done false
30794 9001 Acked Read 1 Done Sent Done false
30795 9002 Acked Write 9 Done Sent Done false
30796 9003 Acked Write 6 Done Sent Done false
30797 9004 Acked Write 4 Done Sent Done false
30798 9005 Acked Read 9 Done Sent Done false
30799 9006 Acked Write 3 Done Sent Done false
30800 9007 Acked Read 8 Done Sent Done false
30801 9008 Acked Write 1 Done Sent Done false
30802 9009 Acked Read 10 Done Sent Done false
30803 9010 Acked Write 8 Done Sent Done false
30804 9011 Acked Read 6 Done Sent Done false
30805 9012 Acked Write 9 Done Sent Done false
30806 9013 Acked Read 8 Done Sent Done false
30807 9014 Acked Flush 0 Done Sent Done false
30808 9015 Acked Write 10 Done Sent Done false
30809 9016 Acked Write 1 Done Sent Done false
30810 9017 Acked Read 9 Done Sent Done false
30811 9018 Acked Write 3 Done Sent Done false
30812 9019 Acked Read 7 Done Sent Done false
30813 9020 Acked Read 7 Done Sent Done false
30814 9021 Acked Write 10 Done Sent Done false
30815 9022 Acked Write 4 Done Sent Done false
30816 9023 Acked Write 4 Done Sent Done false
30817 9024 Acked Write 2 Done Sent Done false
30818 9025 Acked Flush 0 Done Sent Done false
30819 9026 Acked Write 3 Done Sent Done false
30820 9027 Acked Read 5 Done Sent Done false
30821 9028 Acked Read 8 Done Sent Done false
30822 9029 Acked Write 8 Done Sent Done false
30823 9030 Acked Read 6 Done Sent Done false
30824 9031 Acked Flush 0 Done Sent Done false
30825 9032 Acked Read 7 Done Sent Done false
30826 9033 Acked Read 1 Done Sent Done false
30827 9034 Acked Write 10 Done Sent Done false
30828 9035 Acked Read 1 Done Sent Done false
30829 9036 Acked Read 5 Done Sent Done false
30830 9037 Acked Write 5 Done Sent Done false
30831 9038 Acked Write 1 Done Sent Done false
30832 9039 Acked Write 6 Done Sent Done false
30833 9040 Acked Write 10 Done Sent Done false
30834 9041 Acked Write 8 Done Sent Done false
30835 9042 Acked Read 9 Done Sent Done false
30836 9043 Acked Read 5 Done Sent Done false
30837 9044 Acked Read 1 Done Sent Done false
30838 9045 Acked Read 6 Done Sent Done false
30839 9046 Acked Flush 0 Done Sent Done false
30840 9047 Acked Write 4 Done Sent Done false
30841 9048 Acked Write 7 Done Sent Done false
30842 9049 Acked Flush 0 Done Sent Done false
30843 9050 Acked Write 2 Done Sent Done false
30844 9051 Acked Write 2 Done Sent Done false
30845 9052 Acked Write 1 Done Sent Done false
30846 9053 Acked Write 9 Done Sent Done false
30847 9054 Acked Write 5 Done Sent Done false
30848 9055 Acked Read 9 Done Sent Done false
30849 9056 Acked Write 9 Done Sent Done false
30850 9057 Acked Read 8 Done Sent Done false
30851 9058 Acked Read 10 Done Sent Done false
30852 9059 Acked Read 8 Done Sent Done false
30853 9060 Acked Read 3 Done Sent Done false
30854 9061 Acked Write 6 Done Sent Done false
30855 9062 Acked Write 5 Done Sent Done false
30856 9063 Acked Write 3 Done Sent Done false
30857 9064 Acked Read 1 Done Sent Done false
30858 9065 Acked Read 2 Done Sent Done false
30859 9066 Acked Write 4 Done Sent Done false
30860 9067 Acked Read 10 Done Sent Done false
30861 9068 Acked Read 8 Done Sent Done false
30862 9069 Acked Read 4 Done Sent Done false
30863 9070 Acked Write 8 Done Sent Done false
30864 9071 Acked Write 7 Done Sent Done false
30865 9072 Acked Write 10 Done Sent Done false
30866 9073 Acked Write 2 Done Sent Done false
30867 9074 Acked Read 5 Done Sent Done false
30868 9075 Acked Flush 0 Done Sent Done false
30869 9076 Acked Write 5 Done Sent Done false
30870 9077 Acked Write 10 Done Sent Done false
30871 9078 Acked Read 3 Done Sent Done false
30872 9079 Acked Write 10 Done Sent Done false
30873 9080 Acked Flush 0 Done Sent Done false
30874 9081 Acked Write 8 Done Sent Done false
30875 9082 Acked Read 4 Done Sent Done false
30876 9083 Acked Write 6 Done Sent Done false
30877 9084 Acked Write 10 Done Sent Done false
30878 9085 Acked Write 9 Done Sent Done false
30879 9086 Acked Read 9 Done Sent Done false
30880 9087 Acked Write 6 Done Sent Done false
30881 9088 Acked Read 10 Done Sent Done false
30882 9089 Acked Read 5 Done Sent Done false
30883 9090 Acked Write 2 Done Sent Done false
30884 9091 Acked Write 6 Done Sent Done false
30885 9092 Acked Write 1 Done Sent Done false
30886 9093 Acked Read 2 Done Sent Done false
30887 9094 Acked Read 7 Done Sent Done false
30888 9095 Acked Read 10 Done Sent Done false
30889 9096 Acked Flush 0 Done Sent Done false
30890 9097 Acked Read 2 Done Sent Done false
30891 9098 Acked Read 10 Done Sent Done false
30892 9099 Acked Write 6 Done Sent Done false
30893 9100 Acked Read 3 Done Sent Done false
30894 9101 Acked Read 7 Done Sent Done false
30895 9102 Acked Read 3 Done Sent Done false
30896 9103 Acked Read 2 Done Sent Done false
30897 9104 Acked Flush 0 Done Sent Done false
30898 9105 Acked Write 3 Done Sent Done false
30899 9106 Acked Read 10 Done Sent Done false
30900 9107 Acked Write 8 Done Sent Done false
30901 9108 Acked Write 10 Done Sent Done false
30902 9109 Acked Write 7 Done Sent Done false
30903 9110 Acked Write 7 Done Sent Done false
30904 9111 Acked Read 10 Done Sent Done false
30905 9112 Acked Read 6 Done Sent Done false
30906 9113 Acked Read 7 Done Sent Done false
30907 9114 Acked Read 7 Done Sent Done false
30908 9115 Acked Read 2 Done Sent Done false
30909 9116 Acked Read 10 Done Sent Done false
30910 9117 Acked Read 7 Done Sent Done false
30911 9118 Acked Write 8 Done Sent Done false
30912 9119 Acked Read 9 Done Sent Done false
30913 9120 Acked Read 4 Done Sent Done false
30914 9121 Acked Read 8 Done Sent Done false
30915 9122 Acked Read 5 Done Sent Done false
30916 9123 Acked Flush 0 Done Sent Done false
30917 9124 Acked Write 5 Done Sent Done false
30918 9125 Acked Write 4 Done Sent Done false
30919 9126 Acked Write 2 Done Sent Done false
30920 9127 Acked Read 10 Done Sent Done false
30921 9128 Acked Write 3 Done Sent Done false
30922 9129 Acked Write 9 Done Sent Done false
30923 9130 Acked Write 2 Done Sent Done false
30924 9131 Acked Read 7 Done Sent Done false
30925 9132 Acked Write 5 Done Sent Done false
30926 9133 Acked Write 9 Done Sent Done false
30927 9134 Acked Read 6 Done Sent Done false
30928 9135 Acked Read 8 Done Sent Done false
30929 9136 Acked Write 2 Done Sent Done false
30930 9137 Acked Write 3 Done Sent Done false
30931 9138 Acked Read 9 Done Sent Done false
30932 9139 Acked Write 1 Done Sent Done false
30933 9140 Acked Flush 0 Done Sent Done false
30934 9141 Acked Read 10 Done Sent Done false
30935 9142 Acked Write 9 Done Sent Done false
30936 9143 Acked Write 6 Done Sent Done false
30937 9144 Acked Read 5 Done Sent Done false
30938 9145 Acked Write 10 Done Sent Done false
30939 9146 Acked Write 1 Done Sent Done false
30940 9147 Acked Read 1 Done Sent Done false
30941 9148 Acked Write 8 Done Sent Done false
30942 9149 Acked Write 1 Done Sent Done false
30943 9150 Acked Read 1 Done Sent Done false
30944 9151 Acked Write 2 Done Sent Done false
30945 9152 Acked Write 7 Done Sent Done false
30946 9153 Acked Read 9 Done Sent Done false
30947 9154 Acked Write 8 Done Sent Done false
30948 9155 Acked Read 3 Done Sent Done false
30949 9156 Acked Flush 0 Done Sent Done false
30950 9157 Acked Flush 0 Done Sent Done false
30951 9158 Acked Write 3 Done Sent Done false
30952 9159 Acked Read 9 Done Sent Done false
30953 9160 Acked Read 6 Done Sent Done false
30954 9161 Acked Write 2 Done Sent Done false
30955 9162 Acked Write 8 Done Sent Done false
30956 9163 Acked Write 3 Done Sent Done false
30957 9164 Acked Write 6 Done Sent Done false
30958 9165 Acked Flush 0 Done Sent Done false
30959 STATES DS:0 DS:1 DS:2 TOTAL
30960 Sent 0 303 0 303
30961 Done 303 0 303 606
30962 Skipped 0 0 0 0
30963 Error 0 0 0 0
30964 Last Flush: 9165 8862 9165
30965 Downstairs last five completed: 8862 8861 8860 8859 8858
30966 Upstairs last five completed: 9165 9164 9163 9162 9161
30967 CLIENT: Up:0 ds:303 act:2
309682024-11-07T14:09:49.498ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309692024-11-07T14:09:49.498ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45615) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309702024-11-07T14:09:49.499ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309712024-11-07T14:09:49.500ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309722024-11-07T14:09:49.500ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309732024-11-07T14:09:49.500ZINFOcrucible: send last flush ID to this DS: 8862 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309742024-11-07T14:09:49.500ZINFOcrucible: Replied this last flush ID: 8862 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309752024-11-07T14:09:49.500ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309762024-11-07T14:09:49.500ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309772024-11-07T14:09:49.500ZINFOcrucible: [1] client re-new 303 jobs since flush 8862 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309782024-11-07T14:09:49.500ZINFOcrucible: [1] Replayed 303 jobs since flush: 8862 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
30979 ----------------------------------------------------------------
30980 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30981 Downstairs last five completed: 9165 9164 9163 9162 9161
30982 Upstairs last five completed: 9165 9164 9163 9162 9161
30983 CLIENT: Up:0 ds:0 act:3
30984 Replay: All jobs finished
30985 Main task has work to do, go find it
30986 got dsc Stop(1)
30987 stop 1
30988 [8820] Got stop action so:false kr:false
30989 Nov 07 14:09:49.810 INFO request completed, latency_us: 380, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 58ab8048-7182-4589-a66b-a4add7e096a6, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
309902024-11-07T14:09:49.810ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309912024-11-07T14:09:49.811ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309922024-11-07T14:09:49.811ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309932024-11-07T14:09:49.811ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309942024-11-07T14:09:49.811ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
309952024-11-07T14:09:49.811ZINFOcrucible: client task is exiting = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
30996 Nov 07 14:09:49.811 INFO request completed, latency_us: 179, response_code: 200, uri: /state/cid/1, method: GET, req_id: 437b35f3-5a72-4c9d-9c7e-dfb408fb3f06, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
30997 [8820] Exited with: ExitStatus(unix_wait_status(9))
30998 [8820][1] reports Exit
30999 Nov 07 14:09:53.813 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/1, method: GET, req_id: f3cafba3-c2f7-4edd-ab42-177297741ee5, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
31000 001/300 Read block 2391 len 24576
31001 002/300 Read block 2126 len 20480
31002 003/300 Read block 3893 len 32768
31003 004/300 Write block 1231 len 4096 data: 3
31004 005/300 Write block 1708 len 20480 data: 5 5 6 9 7
31005 006/300 Read block 2031 len 24576
31006 007/300 Write block 2881 len 4096 data: 6
31007 008/300 Flush
31008 009/300 Write block 2481 len 8192 data: 6 8
31009 010/300 Flush
31010 011/300 Flush
31011 012/300 Read block 4738 len 24576
31012 013/300 Read block 3202 len 8192
31013 014/300 Read block 3191 len 24576
31014 015/300 Write block 3704 len 4096 data: 8
31015 016/300 Write block 3095 len 20480 data: 4 4 3 3 4
31016 017/300 Write block 525 len 20480 data: 4 4 8 7 7
31017 018/300 Write block 2406 len 40960 data: 4 5 5 5 6 6 6 3 4 6
31018 019/300 Read block 4768 len 28672
31019 020/300 Read block 2534 len 12288
31020 021/300 Write block 1078 len 36864 data: 7 6 7 7 7 7 5 4 4
31021 022/300 Write block 3828 len 32768 data: 9 9 7 6 5 6 6 6
31022 023/300 Write block 1510 len 24576 data: 7 7 7 6 5 4
31023 024/300 Read block 97 len 24576
31024 025/300 Write block 1731 len 4096 data: 2
31025 026/300 Read block 1260 len 24576
31026 027/300 Write block 4848 len 28672 data: 5 4 4 3 3 3 2
31027 028/300 Write block 1651 len 28672 data: 8 7 7 7 6 6 7
31028 029/300 Write block 2859 len 16384 data: 6 6 6 5
31029 030/300 Read block 98 len 4096
31030 031/300 Write block 830 len 12288 data: 9 7 7
31031 032/300 Read block 1685 len 20480
31032 033/300 Write block 3848 len 28672 data: 5 4 5 5 4 4 4
31033 034/300 Read block 2869 len 4096
31034 035/300 Read block 1963 len 24576
31035 036/300 Flush
31036 037/300 Read block 1988 len 8192
31037 038/300 Read block 927 len 4096
31038 039/300 Read block 1768 len 20480
31039 040/300 Write block 1265 len 12288 data: 7 7 7
31040 041/300 Read block 4414 len 32768
31041 042/300 Write block 1916 len 20480 data: 9 7 7 9 8
31042 043/300 Write block 538 len 32768 data: 5 5 4 4 4 6 6 8
31043 044/300 Read block 4915 len 24576
31044 045/300 Read block 3400 len 28672
31045 046/300 Flush
31046 047/300 Read block 188 len 8192
31047 048/300 Read block 2072 len 28672
31048 049/300 Write block 4614 len 32768 data: 4 4 3 3 4 6 7 5
31049 050/300 Flush
31050 051/300 Write block 3382 len 4096 data: 7
31051 052/300 Write block 2809 len 36864 data: 7 5 4 4 5 5 5 7 8
31052 053/300 Write block 623 len 24576 data: 7 5 6 7 7 6
31053 054/300 Read block 1101 len 8192
31054 055/300 Write block 307 len 36864 data: 8 7 6 5 4 6 6 6 4
31055 056/300 Read block 4613 len 16384
31056 057/300 Write block 2040 len 32768 data: 6 6 5 6 6 6 4 4
31057 058/300 Read block 2322 len 32768
31058 059/300 Write block 855 len 32768 data: 8 7 7 6 7 8 7 6
31059 060/300 Flush
31060 061/300 Write block 1032 len 28672 data: 3 2 2 2 3 3 3
31061 062/300 Flush
31062 063/300 Write block 4181 len 20480 data: 7 6 6 5 5
31063 064/300 Write block 2765 len 12288 data: 2 2 3
31064 065/300 Read block 3770 len 16384
31065 066/300 Write block 666 len 8192 data: 3 3
31066 067/300 Read block 4021 len 24576
31067 068/300 Flush
31068 069/300 Read block 2359 len 8192
31069 070/300 Write block 941 len 8192 data: 3 3
31070 071/300 Write block 4446 len 8192 data: 7 7
31071 072/300 Read block 1473 len 12288
31072 073/300 Write block 1823 len 40960 data: 6 7 6 8 9 12 11 12 9 9
31073 074/300 Write block 371 len 8192 data: 4 4
31074 075/300 Write block 666 len 32768 data: 4 4 4 4 5 4 3 3
31075 076/300 Read block 1688 len 24576
31076 077/300 Flush
31077 078/300 Write block 3927 len 8192 data: 10 8
31078 079/300 Read block 4678 len 28672
31079 080/300 Read block 1837 len 28672
31080 081/300 Read block 3261 len 20480
31081 082/300 Read block 1369 len 12288
31082 083/300 Write block 3133 len 32768 data: 6 8 9 9 7 7 6 7
31083 084/300 Read block 1690 len 28672
31084 085/300 Read block 1735 len 40960
31085 086/300 Read block 2688 len 4096
31086 087/300 Write block 206 len 20480 data: 4 4 4 5 6
31087 088/300 Write block 3597 len 20480 data: 4 6 7 7 6
31088 089/300 Read block 1964 len 24576
31089 090/300 Read block 948 len 40960
31090 091/300 Read block 1675 len 16384
31091 092/300 Write block 2571 len 8192 data: 5 3
31092 093/300 Read block 3759 len 4096
31093 094/300 Read block 4430 len 16384
31094 095/300 Flush
31095 096/300 Read block 4796 len 32768
31096 097/300 Read block 4094 len 4096
31097 098/300 Write block 3487 len 36864 data: 8 10 6 7 7 8 9 9 11
31098 099/300 Write block 3117 len 40960 data: 4 5 6 4 4 3 3 5 5 5
31099 100/300 Read block 4722 len 16384
31100 101/300 Write block 3461 len 32768 data: 4 6 7 6 6 4 4 4
31101 102/300 Write block 4513 len 8192 data: 6 6
31102 103/300 Write block 394 len 12288 data: 4 5 4
31103 104/300 Read block 957 len 4096
31104 105/300 Write block 3841 len 32768 data: 7 8 7 6 4 4 6 6
31105 106/300 Write block 237 len 16384 data: 7 7 6 7
31106 107/300 Flush
31107 108/300 Write block 4448 len 8192 data: 6 7
31108 109/300 Write block 2727 len 24576 data: 7 6 5 6 7 10
31109 110/300 Write block 2162 len 8192 data: 6 6
31110 111/300 Write block 3506 len 32768 data: 6 7 7 8 7 8 8 7
31111 112/300 Write block 3777 len 32768 data: 4 4 3 3 3 3 3 5
31112 113/300 Write block 4651 len 12288 data: 5 4 3
31113 114/300 Read block 4980 len 36864
31114 115/300 Flush
31115 116/300 Write block 367 len 36864 data: 5 6 4 4 5 5 4 4 4
31116 117/300 Read block 1705 len 16384
31117 118/300 Write block 3825 len 20480 data: 6 6 6 10 10
31118 119/300 Write block 1738 len 16384 data: 5 5 4 5
31119 120/300 Read block 4545 len 20480
31120 121/300 Read block 3003 len 40960
31121 122/300 Read block 4182 len 4096
31122 123/300 Read block 3878 len 36864
31123 124/300 Read block 4926 len 40960
31124 125/300 Read block 4060 len 40960
31125 126/300 Write block 3527 len 28672 data: 3 3 3 4 5 5 7
31126 127/300 Write block 4191 len 28672 data: 4 5 5 5 4 4 4
31127 128/300 Read block 986 len 24576
31128 129/300 Write block 2218 len 20480 data: 4 3 3 2 3
31129 130/300 Write block 4086 len 12288 data: 6 7 7
31130 131/300 Read block 4656 len 40960
31131 132/300 Write block 1333 len 16384 data: 11 9 10 9
31132 133/300 Write block 3470 len 40960 data: 5 5 6 6 4 4 4 7 8 7
31133 134/300 Flush
31134 135/300 Read block 670 len 32768
31135 136/300 Read block 2435 len 36864
31136 137/300 Write block 1586 len 36864 data: 7 8 6 6 6 6 6 6 7
31137 138/300 Write block 3031 len 12288 data: 7 7 6
31138 139/300 Flush
31139 140/300 Read block 4354 len 40960
31140 141/300 Read block 1541 len 12288
31141 142/300 Read block 251 len 36864
31142 143/300 Write block 1643 len 16384 data: 5 7 6 5
31143 144/300 Read block 3880 len 8192
31144 145/300 Flush
31145 146/300 Write block 4623 len 16384 data: 4 5 5 5
31146 147/300 Read block 1396 len 24576
31147 148/300 Write block 4435 len 28672 data: 6 8 9 10 11 10 11
31148 149/300 Write block 1737 len 28672 data: 4 6 6 5 6 6 5
31149 150/300 Read block 3057 len 12288
31150 151/300 Read block 1578 len 20480
31151 152/300 Write block 429 len 8192 data: 9 8
31152 153/300 Read block 3671 len 16384
31153 154/300 Write block 2637 len 36864 data: 3 4 5 5 7 8 9 7 6
31154 155/300 Read block 612 len 4096
31155 156/300 Write block 2649 len 32768 data: 7 7 6 5 4 4 4 5
31156 157/300 Write block 1836 len 8192 data: 6 6
31157 158/300 Read block 448 len 36864
31158 159/300 Write block 4548 len 28672 data: 4 5 5 4 4 4 5
31159 160/300 Write block 3640 len 28672 data: 6 5 4 4 4 2 4
31160 161/300 Read block 607 len 40960
31161 162/300 Write block 1272 len 40960 data: 7 7 6 6 3 3 5 5 5 5
31162 163/300 Read block 2952 len 32768
31163 164/300 Write block 1753 len 8192 data: 9 9
31164 165/300 Read block 1137 len 40960
31165 166/300 Read block 1503 len 8192
31166 167/300 Write block 1939 len 12288 data: 5 7 7
31167 168/300 Write block 3193 len 24576 data: 3 4 4 7 4 6
31168 169/300 Read block 2611 len 16384
31169 170/300 Write block 4367 len 16384 data: 8 8 8 7
31170 171/300 Read block 3535 len 28672
31171 172/300 Write block 2051 len 8192 data: 5 5
31172 173/300 Write block 4152 len 28672 data: 8 6 5 2 3 4 3
31173 174/300 Read block 1739 len 32768
31174 175/300 Write block 3296 len 8192 data: 9 9
31175 176/300 Read block 1604 len 40960
31176 177/300 Write block 4889 len 8192 data: 2 2
31177 178/300 Write block 3130 len 32768 data: 7 7 6 7 9 10 10 8
31178 179/300 Write block 1163 len 24576 data: 4 5 5 3 4 5
31179 180/300 Read block 3185 len 12288
31180 181/300 Write block 1350 len 8192 data: 6 5
31181 182/300 Write block 216 len 8192 data: 3 3
31182 183/300 Write block 1817 len 4096 data: 4
31183 184/300 Read block 341 len 40960
31184 185/300 Read block 394 len 8192
31185 186/300 Read block 3338 len 8192
31186 187/300 Write block 2929 len 32768 data: 2 2 3 3 3 2 3 6
31187 188/300 Write block 4596 len 24576 data: 7 7 6 6 6 6
31188 189/300 Flush
31189 190/300 Flush
31190 191/300 Flush
31191 192/300 Read block 3670 len 4096
31192 193/300 Read block 4957 len 40960
31193 194/300 Read block 613 len 36864
31194 195/300 Read block 3101 len 40960
31195 196/300 Read block 4142 len 40960
31196 197/300 Read block 3600 len 40960
31197 198/300 Read block 1606 len 24576
31198 199/300 Write block 1944 len 36864 data: 9 8 9 6 7 8 11 9 8
31199 200/300 Read block 2357 len 16384
31200 201/300 Write block 2224 len 36864 data: 3 3 5 5 6 7 7 6 4
31201 202/300 Write block 2706 len 28672 data: 3 3 4 5 6 5 3
31202 203/300 Read block 4478 len 28672
31203 204/300 Write block 2617 len 12288 data: 7 6 6
31204 205/300 Flush
31205 206/300 Read block 4522 len 16384
31206 207/300 Write block 4386 len 20480 data: 6 6 6 6 6
31207 208/300 Write block 1502 len 12288 data: 4 4 5
31208 209/300 Read block 2652 len 12288
31209 210/300 Read block 75 len 24576
31210 211/300 Write block 2675 len 12288 data: 5 4 5
31211 212/300 Write block 322 len 8192 data: 3 4
31212 213/300 Read block 1293 len 40960
31213 214/300 Read block 3891 len 24576
31214 215/300 Write block 4909 len 8192 data: 8 6
31215 216/300 Write block 306 len 24576 data: 6 9 8 7 6 5
31216 217/300 Write block 4564 len 4096 data: 5
31217 218/300 Read block 2680 len 24576
31218 219/300 Write block 4389 len 4096 data: 7
31219 220/300 Read block 4337 len 8192
31220 221/300 Read block 1648 len 24576
31221 222/300 Read block 2962 len 20480
31222 223/300 Write block 2600 len 32768 data: 4 4 5 6 6 5 5 5
31223 224/300 Read block 2227 len 40960
31224 225/300 Read block 4483 len 24576
31225 226/300 Write block 4483 len 40960 data: 9 8 9 8 8 5 3 5 6 5
31226 227/300 Read block 2339 len 4096
31227 228/300 Read block 2518 len 8192
31228 229/300 Write block 2594 len 36864 data: 8 8 8 8 6 7 5 5 6
31229 230/300 Write block 2529 len 24576 data: 5 5 4 5 4 5
31230 231/300 Read block 199 len 4096
31231 232/300 Read block 1980 len 28672
31232 233/300 Read block 2729 len 40960
31233 234/300 Flush
31234 235/300 Read block 775 len 12288
31235 236/300 Write block 4384 len 28672 data: 7 7 7 7 7 8 7
31236 237/300 Write block 3495 len 12288 data: 12 7 7
31237 238/300 Read block 3288 len 40960
31238 239/300 Read block 3147 len 28672
31239 240/300 Write block 3278 len 24576 data: 6 7 7 6 7 8
31240 241/300 Write block 3191 len 36864 data: 3 3 4 5 5 8 5 7 5
31241 242/300 Read block 718 len 36864
31242 243/300 Write block 4572 len 32768 data: 3 4 5 5 5 5 5 5
31243 244/300 Flush
31244 245/300 Read block 2813 len 24576
31245 246/300 Read block 2732 len 12288
31246 247/300 Read block 4480 len 32768
31247 248/300 Read block 2664 len 16384
31248 249/300 Read block 3394 len 12288
31249 250/300 Write block 417 len 8192 data: 7 5
31250 251/300 Write block 178 len 8192 data: 4 7
31251 252/300 Flush
31252 253/300 Write block 1072 len 12288 data: 5 4 5
31253 254/300 Read block 1090 len 16384
31254 255/300 Read block 1355 len 28672
31255 256/300 Read block 2248 len 28672
31256 257/300 Write block 2766 len 16384 data: 3 4 4 6
31257 258/300 Read block 3833 len 32768
31258 259/300 Read block 2455 len 32768
31259 260/300 Write block 4630 len 12288 data: 5 5 5
31260 261/300 Read block 3978 len 20480
31261 262/300 Read block 4457 len 20480
31262 263/300 Read block 1072 len 32768
31263 264/300 Read block 1375 len 16384
31264 265/300 Read block 706 len 4096
31265 266/300 Read block 3299 len 8192
31266 267/300 Write block 115 len 32768 data: 4 4 4 4 5 5 5 6
31267 268/300 Write block 2584 len 12288 data: 5 4 4
31268 269/300 Write block 3018 len 36864 data: 6 6 6 7 8 7 7 6 6
31269 270/300 Read block 614 len 36864
31270 271/300 Write block 2286 len 4096 data: 5
31271 272/300 Write block 4646 len 28672 data: 7 9 10 8 6 6 5
31272 273/300 Write block 2995 len 16384 data: 5 5 7 5
31273 274/300 Flush
31274 275/300 Write block 3268 len 20480 data: 5 7 7 6 7
31275 276/300 Read block 88 len 40960
31276 277/300 Read block 993 len 40960
31277 278/300 Read block 833 len 12288
31278 279/300 Write block 2923 len 28672 data: 4 4 3 3 3 3 3
31279 280/300 Read block 2907 len 12288
31280 281/300 Read block 4649 len 40960
31281 282/300 Read block 3366 len 20480
31282 283/300 Read block 3299 len 32768
31283 284/300 Read block 4189 len 16384
31284 285/300 Write block 4269 len 20480 data: 6 6 9 9 8
31285 286/300 Write block 4702 len 4096 data: 7
31286 287/300 Read block 1732 len 36864
31287 288/300 Read block 908 len 28672
31288 289/300 Read block 3204 len 36864
31289 290/300 Write block 166 len 24576 data: 4 4 5 5 4 4
31290 291/300 Read block 3567 len 36864
31291 292/300 Read block 387 len 28672
31292 293/300 Write block 3099 len 4096 data: 5
31293 294/300 Read block 503 len 4096
31294 295/300 Read block 3628 len 36864
31295 296/300 Write block 4308 len 40960 data: 6 9 8 9 8 8 9 7 6 6
31296 297/300 Flush
31297 298/300 Write block 1588 len 20480 data: 7 7 7 7 7
31298 299/300 Write block 1066 len 36864 data: 5 3 3 3 3 4 6 5 6
31299 300/300 Write block 2071 len 24576 data: 3 2 2 3 4 5
31300 301/300 Flush
31301 Main task has work to do, go find it
31302 got dsc Start(1)
31303 start 1
31304 Nov 07 14:09:54.100 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/1, method: POST, req_id: cc492611-1a55-467d-9a3c-8afa8cee3240, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
31305 Replay: started 1, returned:Ok(())
31306 ----------------------------------------------------------------
31307 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31308 States: Active Offline Active
31309 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31310 9166 Acked Read 6 Done Sent Done false
31311 9167 Acked Read 5 Done Sent Done false
31312 9168 Acked Read 8 Done Sent Done false
31313 9169 Acked Write 1 Done Sent Done false
31314 9170 Acked Write 5 Done Sent Done false
31315 9171 Acked Read 6 Done Sent Done false
31316 9172 Acked Write 1 Done Sent Done false
31317 9173 Acked Flush 0 Done Sent Done false
31318 9174 Acked Write 2 Done Sent Done false
31319 9175 Acked Flush 0 Done Sent Done false
31320 9176 Acked Flush 0 Done Sent Done false
31321 9177 Acked Read 6 Done Sent Done false
31322 9178 Acked Read 2 Done Sent Done false
31323 9179 Acked Read 6 Done Sent Done false
31324 9180 Acked Write 1 Done Sent Done false
31325 9181 Acked Write 5 Done Sent Done false
31326 9182 Acked Write 5 Done Sent Done false
31327 9183 Acked Write 10 Done Sent Done false
31328 9184 Acked Read 7 Done Sent Done false
31329 9185 Acked Read 3 Done Sent Done false
31330 9186 Acked Write 9 Done Sent Done false
31331 9187 Acked Write 8 Done Sent Done false
31332 9188 Acked Write 6 Done Sent Done false
31333 9189 Acked Read 6 Done Sent Done false
31334 9190 Acked Write 1 Done Sent Done false
31335 9191 Acked Read 6 Done Sent Done false
31336 9192 Acked Write 7 Done Sent Done false
31337 9193 Acked Write 7 Done Sent Done false
31338 9194 Acked Write 4 Done Sent Done false
31339 9195 Acked Read 1 Done Sent Done false
31340 9196 Acked Write 3 Done Sent Done false
31341 9197 Acked Read 5 Done Sent Done false
31342 9198 Acked Write 7 Done Sent Done false
31343 9199 Acked Read 1 Done Sent Done false
31344 9200 Acked Read 6 Done Sent Done false
31345 9201 Acked Flush 0 Done Sent Done false
31346 9202 Acked Read 2 Done Sent Done false
31347 9203 Acked Read 1 Done Sent Done false
31348 9204 Acked Read 5 Done Sent Done false
31349 9205 Acked Write 3 Done Sent Done false
31350 9206 Acked Read 8 Done Sent Done false
31351 9207 Acked Write 5 Done Sent Done false
31352 9208 Acked Write 8 Done Sent Done false
31353 9209 Acked Read 6 Done Sent Done false
31354 9210 Acked Read 7 Done Sent Done false
31355 9211 Acked Flush 0 Done Sent Done false
31356 9212 Acked Read 2 Done Sent Done false
31357 9213 Acked Read 7 Done Sent Done false
31358 9214 Acked Write 8 Done Sent Done false
31359 9215 Acked Flush 0 Done Sent Done false
31360 9216 Acked Write 1 Done Sent Done false
31361 9217 Acked Write 9 Done Sent Done false
31362 9218 Acked Write 6 Done Sent Done false
31363 9219 Acked Read 2 Done Sent Done false
31364 9220 Acked Write 9 Done Sent Done false
31365 9221 Acked Read 4 Done Sent Done false
31366 9222 Acked Write 8 Done Sent Done false
31367 9223 Acked Read 8 Done Sent Done false
31368 9224 Acked Write 8 Done Sent Done false
31369 9225 Acked Flush 0 Done Sent Done false
31370 9226 Acked Write 7 Done Sent Done false
31371 9227 Acked Flush 0 Done Sent Done false
31372 9228 Acked Write 5 Done Sent Done false
31373 9229 Acked Write 3 Done Sent Done false
31374 9230 Acked Read 4 Done Sent Done false
31375 9231 Acked Write 2 Done Sent Done false
31376 9232 Acked Read 6 Done Sent Done false
31377 9233 Acked Flush 0 Done Sent Done false
31378 9234 Acked Read 2 Done Sent Done false
31379 9235 Acked Write 2 Done Sent Done false
31380 9236 Acked Write 2 Done Sent Done false
31381 9237 Acked Read 3 Done Sent Done false
31382 9238 Acked Write 10 Done Sent Done false
31383 9239 Acked Write 2 Done Sent Done false
31384 9240 Acked Write 8 Done Sent Done false
31385 9241 Acked Read 6 Done Sent Done false
31386 9242 Acked Flush 0 Done Sent Done false
31387 9243 Acked Write 2 Done Sent Done false
31388 9244 Acked Read 7 Done Sent Done false
31389 9245 Acked Read 7 Done Sent Done false
31390 9246 Acked Read 5 Done Sent Done false
31391 9247 Acked Read 3 Done Sent Done false
31392 9248 Acked Write 8 Done Sent Done false
31393 9249 Acked Read 7 Done Sent Done false
31394 9250 Acked Read 10 Done Sent Done false
31395 9251 Acked Read 1 Done Sent Done false
31396 9252 Acked Write 5 Done Sent Done false
31397 9253 Acked Write 5 Done Sent Done false
31398 9254 Acked Read 6 Done Sent Done false
31399 9255 Acked Read 10 Done Sent Done false
31400 9256 Acked Read 4 Done Sent Done false
31401 9257 Acked Write 2 Done Sent Done false
31402 9258 Acked Read 1 Done Sent Done false
31403 9259 Acked Read 4 Done Sent Done false
31404 9260 Acked Flush 0 Done Sent Done false
31405 9261 Acked Read 8 Done Sent Done false
31406 9262 Acked Read 1 Done Sent Done false
31407 9263 Acked Write 9 Done Sent Done false
31408 9264 Acked Write 10 Done Sent Done false
31409 9265 Acked Read 4 Done Sent Done false
31410 9266 Acked Write 8 Done Sent Done false
31411 9267 Acked Write 2 Done Sent Done false
31412 9268 Acked Write 3 Done Sent Done false
31413 9269 Acked Read 1 Done Sent Done false
31414 9270 Acked Write 8 Done Sent Done false
31415 9271 Acked Write 4 Done Sent Done false
31416 9272 Acked Flush 0 Done Sent Done false
31417 9273 Acked Write 2 Done Sent Done false
31418 9274 Acked Write 6 Done Sent Done false
31419 9275 Acked Write 2 Done Sent Done false
31420 9276 Acked Write 8 Done Sent Done false
31421 9277 Acked Write 8 Done Sent Done false
31422 9278 Acked Write 3 Done Sent Done false
31423 9279 Acked Read 9 Done Sent Done false
31424 9280 Acked Flush 0 Done Sent Done false
31425 9281 Acked Write 9 Done Sent Done false
31426 9282 Acked Read 4 Done Sent Done false
31427 9283 Acked Write 5 Done Sent Done false
31428 9284 Acked Write 4 Done Sent Done false
31429 9285 Acked Read 5 Done Sent Done false
31430 9286 Acked Read 10 Done Sent Done false
31431 9287 Acked Read 1 Done Sent Done false
31432 9288 Acked Read 9 Done Sent Done false
31433 9289 Acked Read 10 Done Sent Done false
31434 9290 Acked Read 10 Done Sent Done false
31435 9291 Acked Write 7 Done Sent Done false
31436 9292 Acked Write 7 Done Sent Done false
31437 9293 Acked Read 6 Done Sent Done false
31438 9294 Acked Write 5 Done Sent Done false
31439 9295 Acked Write 3 Done Sent Done false
31440 9296 Acked Read 10 Done Sent Done false
31441 9297 Acked Write 4 Done Sent Done false
31442 9298 Acked Write 10 Done Sent Done false
31443 9299 Acked Flush 0 Done Sent Done false
31444 9300 Acked Read 8 Done Sent Done false
31445 9301 Acked Read 9 Done Sent Done false
31446 9302 Acked Write 9 Done Sent Done false
31447 9303 Acked Write 3 Done Sent Done false
31448 9304 Acked Flush 0 Done Sent Done false
31449 9305 Acked Read 10 Done Sent Done false
31450 9306 Acked Read 3 Done Sent Done false
31451 9307 Acked Read 9 Done Sent Done false
31452 9308 Acked Write 4 Done Sent Done false
31453 9309 Acked Read 2 Done Sent Done false
31454 9310 Acked Flush 0 Done Sent Done false
31455 9311 Acked Write 4 Done Sent Done false
31456 9312 Acked Read 6 Done Sent Done false
31457 9313 Acked Write 7 Done Sent Done false
31458 9314 Acked Write 7 Done Sent Done false
31459 9315 Acked Read 3 Done Sent Done false
31460 9316 Acked Read 5 Done Sent Done false
31461 9317 Acked Write 2 Done Sent Done false
31462 9318 Acked Read 4 Done Sent Done false
31463 9319 Acked Write 9 Done Sent Done false
31464 9320 Acked Read 1 Done Sent Done false
31465 9321 Acked Write 8 Done Sent Done false
31466 9322 Acked Write 2 Done Sent Done false
31467 9323 Acked Read 9 Done Sent Done false
31468 9324 Acked Write 7 Done Sent Done false
31469 9325 Acked Write 7 Done Sent Done false
31470 9326 Acked Read 10 Done Sent Done false
31471 9327 Acked Write 10 Done Sent Done false
31472 9328 Acked Read 8 Done Sent Done false
31473 9329 Acked Write 2 Done Sent Done false
31474 9330 Acked Read 10 Done Sent Done false
31475 9331 Acked Read 2 Done Sent Done false
31476 9332 Acked Write 3 Done Sent Done false
31477 9333 Acked Write 6 Done Sent Done false
31478 9334 Acked Read 4 Done Sent Done false
31479 9335 Acked Write 4 Done Sent Done false
31480 9336 Acked Read 7 Done Sent Done false
31481 9337 Acked Write 2 Done Sent Done false
31482 9338 Acked Write 7 Done Sent Done false
31483 9339 Acked Read 8 Done Sent Done false
31484 9340 Acked Write 2 Done Sent Done false
31485 9341 Acked Read 10 Done Sent Done false
31486 9342 Acked Write 2 Done Sent Done false
31487 9343 Acked Write 8 Done Sent Done false
31488 9344 Acked Write 6 Done Sent Done false
31489 9345 Acked Read 3 Done Sent Done false
31490 9346 Acked Write 2 Done Sent Done false
31491 9347 Acked Write 2 Done Sent Done false
31492 9348 Acked Write 1 Done Sent Done false
31493 9349 Acked Read 10 Done Sent Done false
31494 9350 Acked Read 2 Done Sent Done false
31495 9351 Acked Read 2 Done Sent Done false
31496 9352 Acked Write 8 Done Sent Done false
31497 9353 Acked Write 6 Done Sent Done false
31498 9354 Acked Flush 0 Done Sent Done false
31499 9355 Acked Flush 0 Done Sent Done false
31500 9356 Acked Flush 0 Done Sent Done false
31501 9357 Acked Read 1 Done Sent Done false
31502 9358 Acked Read 10 Done Sent Done false
31503 9359 Acked Read 9 Done Sent Done false
31504 9360 Acked Read 10 Done Sent Done false
31505 9361 Acked Read 10 Done Sent Done false
31506 9362 Acked Read 10 Done Sent Done false
31507 9363 Acked Read 6 Done Sent Done false
31508 9364 Acked Write 9 Done Sent Done false
31509 9365 Acked Read 4 Done Sent Done false
31510 9366 Acked Write 9 Done Sent Done false
31511 9367 Acked Write 7 Done Sent Done false
31512 9368 Acked Read 7 Done Sent Done false
31513 9369 Acked Write 3 Done Sent Done false
31514 9370 Acked Flush 0 Done Sent Done false
31515 9371 Acked Read 4 Done Sent Done false
31516 9372 Acked Write 5 Done Sent Done false
31517 9373 Acked Write 3 Done Sent Done false
31518 9374 Acked Read 3 Done Sent Done false
31519 9375 Acked Read 6 Done Sent Done false
31520 9376 Acked Write 3 Done Sent Done false
31521 9377 Acked Write 2 Done Sent Done false
31522 9378 Acked Read 10 Done Sent Done false
31523 9379 Acked Read 6 Done Sent Done false
31524 9380 Acked Write 2 Done Sent Done false
31525 9381 Acked Write 6 Done Sent Done false
31526 9382 Acked Write 1 Done Sent Done false
31527 9383 Acked Read 6 Done Sent Done false
31528 9384 Acked Write 1 Done Sent Done false
31529 9385 Acked Read 2 Done Sent Done false
31530 9386 Acked Read 6 Done Sent Done false
31531 9387 Acked Read 5 Done Sent Done false
31532 9388 Acked Write 8 Done Sent Done false
31533 9389 Acked Read 10 Done Sent Done false
31534 9390 Acked Read 6 Done Sent Done false
31535 9391 Acked Write 10 Done Sent Done false
31536 9392 Acked Read 1 Done Sent Done false
31537 9393 Acked Read 2 Done Sent Done false
31538 9394 Acked Write 9 Done Sent Done false
31539 9395 Acked Write 6 Done Sent Done false
31540 9396 Acked Read 1 Done Sent Done false
31541 9397 Acked Read 7 Done Sent Done false
31542 9398 Acked Read 10 Done Sent Done false
31543 9399 Acked Flush 0 Done Sent Done false
31544 9400 Acked Read 3 Done Sent Done false
31545 9401 Acked Write 7 Done Sent Done false
31546 9402 Acked Write 3 Done Sent Done false
31547 9403 Acked Read 10 Done Sent Done false
31548 9404 Acked Read 7 Done Sent Done false
31549 9405 Acked Write 6 Done Sent Done false
31550 9406 Acked Write 9 Done Sent Done false
31551 9407 Acked Read 9 Done Sent Done false
31552 9408 Acked Write 8 Done Sent Done false
31553 9409 Acked Flush 0 Done Sent Done false
31554 9410 Acked Read 6 Done Sent Done false
31555 9411 Acked Read 3 Done Sent Done false
31556 9412 Acked Read 8 Done Sent Done false
31557 9413 Acked Read 4 Done Sent Done false
31558 9414 Acked Flush 0 Done Sent Done false
31559 9415 Acked Read 3 Done Sent Done false
31560 9416 Acked Write 2 Done Sent Done false
31561 9417 Acked Write 2 Done Sent Done false
31562 9418 Acked Flush 0 Done Sent Done false
31563 9419 Acked Write 3 Done Sent Done false
31564 9420 Acked Read 4 Done Sent Done false
31565 9421 Acked Read 7 Done Sent Done false
31566 9422 Acked Read 7 Done Sent Done false
31567 9423 Acked Write 4 Done Sent Done false
31568 9424 Acked Read 8 Done Sent Done false
31569 9425 Acked Read 8 Done Sent Done false
31570 9426 Acked Write 3 Done Sent Done false
31571 9427 Acked Read 5 Done Sent Done false
31572 9428 Acked Read 5 Done Sent Done false
31573 9429 Acked Read 8 Done Sent Done false
31574 9430 Acked Read 4 Done Sent Done false
31575 9431 Acked Read 1 Done Sent Done false
31576 9432 Acked Read 2 Done Sent Done false
31577 9433 Acked Write 8 Done Sent Done false
31578 9434 Acked Write 3 Done Sent Done false
31579 9435 Acked Write 9 Done Sent Done false
31580 9436 Acked Read 9 Done Sent Done false
31581 9437 Acked Write 1 Done Sent Done false
31582 9438 Acked Write 7 Done Sent Done false
31583 9439 Acked Write 4 Done Sent Done false
31584 9440 Acked Flush 0 Done Sent Done false
31585 9441 Acked Write 5 Done Sent Done false
31586 9442 Acked Read 10 Done Sent Done false
31587 9443 Acked Read 10 Done Sent Done false
31588 9444 Acked Read 3 Done Sent Done false
31589 9445 Acked Write 7 Done Sent Done false
31590 9446 Acked Read 3 Done Sent Done false
31591 9447 Acked Read 10 Done Sent Done false
31592 9448 Acked Read 5 Done Sent Done false
31593 9449 Acked Read 8 Done Sent Done false
31594 9450 Acked Read 4 Done Sent Done false
31595 9451 Acked Write 5 Done Sent Done false
31596 9452 Acked Write 1 Done Sent Done false
31597 9453 Acked Read 9 Done Sent Done false
31598 9454 Acked Read 7 Done Sent Done false
31599 9455 Acked Read 9 Done Sent Done false
31600 9456 Acked Write 6 Done Sent Done false
31601 9457 Acked Read 9 Done Sent Done false
31602 9458 Acked Read 7 Done Sent Done false
31603 9459 Acked Write 1 Done Sent Done false
31604 9460 Acked Read 1 Done Sent Done false
31605 9461 Acked Read 9 Done Sent Done false
31606 9462 Acked Write 10 Done Sent Done false
31607 9463 Acked Flush 0 Done Sent Done false
31608 9464 Acked Write 5 Done Sent Done false
31609 9465 Acked Write 9 Done Sent Done false
31610 9466 Acked Write 6 Done Sent Done false
31611 9467 Acked Flush 0 Done Sent Done false
31612 STATES DS:0 DS:1 DS:2 TOTAL
31613 Sent 0 302 0 302
31614 Done 302 0 302 604
31615 Skipped 0 0 0 0
31616 Error 0 0 0 0
31617 Last Flush: 9467 9165 9467
31618 Downstairs last five completed: 9165 9164 9163 9162 9161
31619 Upstairs last five completed: 9467 9466 9465 9464 9463
31620 CLIENT: Up:0 ds:302 act:2
31621 [8820] got start action
31622 [8820] I am going to restart
31623 Starting downstairs at port 8820
31624 Make output file at "/tmp/dsc/downstairs-8820.txt"
31625 [8820][1] reports Starting
31626 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1227)
31627 [8820][1] reports Running
31628 ----------------------------------------------------------------
31629 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31630 States: Active Offline Active
31631 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31632 9166 Acked Read 6 Done Sent Done false
31633 9167 Acked Read 5 Done Sent Done false
31634 9168 Acked Read 8 Done Sent Done false
31635 9169 Acked Write 1 Done Sent Done false
31636 9170 Acked Write 5 Done Sent Done false
31637 9171 Acked Read 6 Done Sent Done false
31638 9172 Acked Write 1 Done Sent Done false
31639 9173 Acked Flush 0 Done Sent Done false
31640 9174 Acked Write 2 Done Sent Done false
31641 9175 Acked Flush 0 Done Sent Done false
31642 9176 Acked Flush 0 Done Sent Done false
31643 9177 Acked Read 6 Done Sent Done false
31644 9178 Acked Read 2 Done Sent Done false
31645 9179 Acked Read 6 Done Sent Done false
31646 9180 Acked Write 1 Done Sent Done false
31647 9181 Acked Write 5 Done Sent Done false
31648 9182 Acked Write 5 Done Sent Done false
31649 9183 Acked Write 10 Done Sent Done false
31650 9184 Acked Read 7 Done Sent Done false
31651 9185 Acked Read 3 Done Sent Done false
31652 9186 Acked Write 9 Done Sent Done false
31653 9187 Acked Write 8 Done Sent Done false
31654 9188 Acked Write 6 Done Sent Done false
31655 9189 Acked Read 6 Done Sent Done false
31656 9190 Acked Write 1 Done Sent Done false
31657 9191 Acked Read 6 Done Sent Done false
31658 9192 Acked Write 7 Done Sent Done false
31659 9193 Acked Write 7 Done Sent Done false
31660 9194 Acked Write 4 Done Sent Done false
31661 9195 Acked Read 1 Done Sent Done false
31662 9196 Acked Write 3 Done Sent Done false
31663 9197 Acked Read 5 Done Sent Done false
31664 9198 Acked Write 7 Done Sent Done false
31665 9199 Acked Read 1 Done Sent Done false
31666 9200 Acked Read 6 Done Sent Done false
31667 9201 Acked Flush 0 Done Sent Done false
31668 9202 Acked Read 2 Done Sent Done false
31669 9203 Acked Read 1 Done Sent Done false
31670 9204 Acked Read 5 Done Sent Done false
31671 9205 Acked Write 3 Done Sent Done false
31672 9206 Acked Read 8 Done Sent Done false
31673 9207 Acked Write 5 Done Sent Done false
31674 9208 Acked Write 8 Done Sent Done false
31675 9209 Acked Read 6 Done Sent Done false
31676 9210 Acked Read 7 Done Sent Done false
31677 9211 Acked Flush 0 Done Sent Done false
31678 9212 Acked Read 2 Done Sent Done false
31679 9213 Acked Read 7 Done Sent Done false
31680 9214 Acked Write 8 Done Sent Done false
31681 9215 Acked Flush 0 Done Sent Done false
31682 9216 Acked Write 1 Done Sent Done false
31683 9217 Acked Write 9 Done Sent Done false
31684 9218 Acked Write 6 Done Sent Done false
31685 9219 Acked Read 2 Done Sent Done false
31686 9220 Acked Write 9 Done Sent Done false
31687 9221 Acked Read 4 Done Sent Done false
31688 9222 Acked Write 8 Done Sent Done false
31689 9223 Acked Read 8 Done Sent Done false
31690 9224 Acked Write 8 Done Sent Done false
31691 9225 Acked Flush 0 Done Sent Done false
31692 9226 Acked Write 7 Done Sent Done false
31693 9227 Acked Flush 0 Done Sent Done false
31694 9228 Acked Write 5 Done Sent Done false
31695 9229 Acked Write 3 Done Sent Done false
31696 9230 Acked Read 4 Done Sent Done false
31697 9231 Acked Write 2 Done Sent Done false
31698 9232 Acked Read 6 Done Sent Done false
31699 9233 Acked Flush 0 Done Sent Done false
31700 9234 Acked Read 2 Done Sent Done false
31701 9235 Acked Write 2 Done Sent Done false
31702 9236 Acked Write 2 Done Sent Done false
31703 9237 Acked Read 3 Done Sent Done false
31704 9238 Acked Write 10 Done Sent Done false
31705 9239 Acked Write 2 Done Sent Done false
31706 9240 Acked Write 8 Done Sent Done false
31707 9241 Acked Read 6 Done Sent Done false
31708 9242 Acked Flush 0 Done Sent Done false
31709 9243 Acked Write 2 Done Sent Done false
31710 9244 Acked Read 7 Done Sent Done false
31711 9245 Acked Read 7 Done Sent Done false
31712 9246 Acked Read 5 Done Sent Done false
31713 9247 Acked Read 3 Done Sent Done false
31714 9248 Acked Write 8 Done Sent Done false
31715 9249 Acked Read 7 Done Sent Done false
31716 9250 Acked Read 10 Done Sent Done false
31717 9251 Acked Read 1 Done Sent Done false
31718 9252 Acked Write 5 Done Sent Done false
31719 9253 Acked Write 5 Done Sent Done false
31720 9254 Acked Read 6 Done Sent Done false
31721 9255 Acked Read 10 Done Sent Done false
31722 9256 Acked Read 4 Done Sent Done false
31723 9257 Acked Write 2 Done Sent Done false
31724 9258 Acked Read 1 Done Sent Done false
31725 9259 Acked Read 4 Done Sent Done false
31726 9260 Acked Flush 0 Done Sent Done false
31727 9261 Acked Read 8 Done Sent Done false
31728 9262 Acked Read 1 Done Sent Done false
31729 9263 Acked Write 9 Done Sent Done false
31730 9264 Acked Write 10 Done Sent Done false
31731 9265 Acked Read 4 Done Sent Done false
31732 9266 Acked Write 8 Done Sent Done false
31733 9267 Acked Write 2 Done Sent Done false
31734 9268 Acked Write 3 Done Sent Done false
31735 9269 Acked Read 1 Done Sent Done false
31736 9270 Acked Write 8 Done Sent Done false
31737 9271 Acked Write 4 Done Sent Done false
31738 9272 Acked Flush 0 Done Sent Done false
31739 9273 Acked Write 2 Done Sent Done false
31740 9274 Acked Write 6 Done Sent Done false
31741 9275 Acked Write 2 Done Sent Done false
31742 9276 Acked Write 8 Done Sent Done false
31743 9277 Acked Write 8 Done Sent Done false
31744 9278 Acked Write 3 Done Sent Done false
31745 9279 Acked Read 9 Done Sent Done false
31746 9280 Acked Flush 0 Done Sent Done false
31747 9281 Acked Write 9 Done Sent Done false
31748 9282 Acked Read 4 Done Sent Done false
31749 9283 Acked Write 5 Done Sent Done false
31750 9284 Acked Write 4 Done Sent Done false
31751 9285 Acked Read 5 Done Sent Done false
31752 9286 Acked Read 10 Done Sent Done false
31753 9287 Acked Read 1 Done Sent Done false
31754 9288 Acked Read 9 Done Sent Done false
31755 9289 Acked Read 10 Done Sent Done false
31756 9290 Acked Read 10 Done Sent Done false
31757 9291 Acked Write 7 Done Sent Done false
31758 9292 Acked Write 7 Done Sent Done false
31759 9293 Acked Read 6 Done Sent Done false
31760 9294 Acked Write 5 Done Sent Done false
31761 9295 Acked Write 3 Done Sent Done false
31762 9296 Acked Read 10 Done Sent Done false
31763 9297 Acked Write 4 Done Sent Done false
31764 9298 Acked Write 10 Done Sent Done false
31765 9299 Acked Flush 0 Done Sent Done false
31766 9300 Acked Read 8 Done Sent Done false
31767 9301 Acked Read 9 Done Sent Done false
31768 9302 Acked Write 9 Done Sent Done false
31769 9303 Acked Write 3 Done Sent Done false
31770 9304 Acked Flush 0 Done Sent Done false
31771 9305 Acked Read 10 Done Sent Done false
31772 9306 Acked Read 3 Done Sent Done false
31773 9307 Acked Read 9 Done Sent Done false
31774 9308 Acked Write 4 Done Sent Done false
31775 9309 Acked Read 2 Done Sent Done false
31776 9310 Acked Flush 0 Done Sent Done false
31777 9311 Acked Write 4 Done Sent Done false
31778 9312 Acked Read 6 Done Sent Done false
31779 9313 Acked Write 7 Done Sent Done false
31780 9314 Acked Write 7 Done Sent Done false
31781 9315 Acked Read 3 Done Sent Done false
31782 9316 Acked Read 5 Done Sent Done false
31783 9317 Acked Write 2 Done Sent Done false
31784 9318 Acked Read 4 Done Sent Done false
31785 9319 Acked Write 9 Done Sent Done false
31786 9320 Acked Read 1 Done Sent Done false
31787 9321 Acked Write 8 Done Sent Done false
31788 9322 Acked Write 2 Done Sent Done false
31789 9323 Acked Read 9 Done Sent Done false
31790 9324 Acked Write 7 Done Sent Done false
31791 9325 Acked Write 7 Done Sent Done false
31792 9326 Acked Read 10 Done Sent Done false
31793 9327 Acked Write 10 Done Sent Done false
31794 9328 Acked Read 8 Done Sent Done false
31795 9329 Acked Write 2 Done Sent Done false
31796 9330 Acked Read 10 Done Sent Done false
31797 9331 Acked Read 2 Done Sent Done false
31798 9332 Acked Write 3 Done Sent Done false
31799 9333 Acked Write 6 Done Sent Done false
31800 9334 Acked Read 4 Done Sent Done false
31801 9335 Acked Write 4 Done Sent Done false
31802 9336 Acked Read 7 Done Sent Done false
31803 9337 Acked Write 2 Done Sent Done false
31804 9338 Acked Write 7 Done Sent Done false
31805 9339 Acked Read 8 Done Sent Done false
31806 9340 Acked Write 2 Done Sent Done false
31807 9341 Acked Read 10 Done Sent Done false
31808 9342 Acked Write 2 Done Sent Done false
31809 9343 Acked Write 8 Done Sent Done false
31810 9344 Acked Write 6 Done Sent Done false
31811 9345 Acked Read 3 Done Sent Done false
31812 9346 Acked Write 2 Done Sent Done false
31813 9347 Acked Write 2 Done Sent Done false
31814 9348 Acked Write 1 Done Sent Done false
31815 9349 Acked Read 10 Done Sent Done false
31816 9350 Acked Read 2 Done Sent Done false
31817 9351 Acked Read 2 Done Sent Done false
31818 9352 Acked Write 8 Done Sent Done false
31819 9353 Acked Write 6 Done Sent Done false
31820 9354 Acked Flush 0 Done Sent Done false
31821 9355 Acked Flush 0 Done Sent Done false
31822 9356 Acked Flush 0 Done Sent Done false
31823 9357 Acked Read 1 Done Sent Done false
31824 9358 Acked Read 10 Done Sent Done false
31825 9359 Acked Read 9 Done Sent Done false
31826 9360 Acked Read 10 Done Sent Done false
31827 9361 Acked Read 10 Done Sent Done false
31828 9362 Acked Read 10 Done Sent Done false
31829 9363 Acked Read 6 Done Sent Done false
31830 9364 Acked Write 9 Done Sent Done false
31831 9365 Acked Read 4 Done Sent Done false
31832 9366 Acked Write 9 Done Sent Done false
31833 9367 Acked Write 7 Done Sent Done false
31834 9368 Acked Read 7 Done Sent Done false
31835 9369 Acked Write 3 Done Sent Done false
31836 9370 Acked Flush 0 Done Sent Done false
31837 9371 Acked Read 4 Done Sent Done false
31838 9372 Acked Write 5 Done Sent Done false
31839 9373 Acked Write 3 Done Sent Done false
31840 9374 Acked Read 3 Done Sent Done false
31841 9375 Acked Read 6 Done Sent Done false
31842 9376 Acked Write 3 Done Sent Done false
31843 9377 Acked Write 2 Done Sent Done false
31844 9378 Acked Read 10 Done Sent Done false
31845 9379 Acked Read 6 Done Sent Done false
31846 9380 Acked Write 2 Done Sent Done false
31847 9381 Acked Write 6 Done Sent Done false
31848 9382 Acked Write 1 Done Sent Done false
31849 9383 Acked Read 6 Done Sent Done false
31850 9384 Acked Write 1 Done Sent Done false
31851 9385 Acked Read 2 Done Sent Done false
31852 9386 Acked Read 6 Done Sent Done false
31853 9387 Acked Read 5 Done Sent Done false
31854 9388 Acked Write 8 Done Sent Done false
31855 9389 Acked Read 10 Done Sent Done false
31856 9390 Acked Read 6 Done Sent Done false
31857 9391 Acked Write 10 Done Sent Done false
31858 9392 Acked Read 1 Done Sent Done false
31859 9393 Acked Read 2 Done Sent Done false
31860 9394 Acked Write 9 Done Sent Done false
31861 9395 Acked Write 6 Done Sent Done false
31862 9396 Acked Read 1 Done Sent Done false
31863 9397 Acked Read 7 Done Sent Done false
31864 9398 Acked Read 10 Done Sent Done false
31865 9399 Acked Flush 0 Done Sent Done false
31866 9400 Acked Read 3 Done Sent Done false
31867 9401 Acked Write 7 Done Sent Done false
31868 9402 Acked Write 3 Done Sent Done false
31869 9403 Acked Read 10 Done Sent Done false
31870 9404 Acked Read 7 Done Sent Done false
31871 9405 Acked Write 6 Done Sent Done false
31872 9406 Acked Write 9 Done Sent Done false
31873 9407 Acked Read 9 Done Sent Done false
31874 9408 Acked Write 8 Done Sent Done false
31875 9409 Acked Flush 0 Done Sent Done false
31876 9410 Acked Read 6 Done Sent Done false
31877 9411 Acked Read 3 Done Sent Done false
31878 9412 Acked Read 8 Done Sent Done false
31879 9413 Acked Read 4 Done Sent Done false
31880 9414 Acked Flush 0 Done Sent Done false
31881 9415 Acked Read 3 Done Sent Done false
31882 9416 Acked Write 2 Done Sent Done false
31883 9417 Acked Write 2 Done Sent Done false
31884 9418 Acked Flush 0 Done Sent Done false
31885 9419 Acked Write 3 Done Sent Done false
31886 9420 Acked Read 4 Done Sent Done false
31887 9421 Acked Read 7 Done Sent Done false
31888 9422 Acked Read 7 Done Sent Done false
31889 9423 Acked Write 4 Done Sent Done false
31890 9424 Acked Read 8 Done Sent Done false
31891 9425 Acked Read 8 Done Sent Done false
31892 9426 Acked Write 3 Done Sent Done false
31893 9427 Acked Read 5 Done Sent Done false
31894 9428 Acked Read 5 Done Sent Done false
31895 9429 Acked Read 8 Done Sent Done false
31896 9430 Acked Read 4 Done Sent Done false
31897 9431 Acked Read 1 Done Sent Done false
31898 9432 Acked Read 2 Done Sent Done false
31899 9433 Acked Write 8 Done Sent Done false
31900 9434 Acked Write 3 Done Sent Done false
31901 9435 Acked Write 9 Done Sent Done false
31902 9436 Acked Read 9 Done Sent Done false
31903 9437 Acked Write 1 Done Sent Done false
31904 9438 Acked Write 7 Done Sent Done false
31905 9439 Acked Write 4 Done Sent Done false
31906 9440 Acked Flush 0 Done Sent Done false
31907 9441 Acked Write 5 Done Sent Done false
31908 9442 Acked Read 10 Done Sent Done false
31909 9443 Acked Read 10 Done Sent Done false
31910 9444 Acked Read 3 Done Sent Done false
31911 9445 Acked Write 7 Done Sent Done false
31912 9446 Acked Read 3 Done Sent Done false
31913 9447 Acked Read 10 Done Sent Done false
31914 9448 Acked Read 5 Done Sent Done false
31915 9449 Acked Read 8 Done Sent Done false
31916 9450 Acked Read 4 Done Sent Done false
31917 9451 Acked Write 5 Done Sent Done false
31918 9452 Acked Write 1 Done Sent Done false
31919 9453 Acked Read 9 Done Sent Done false
31920 9454 Acked Read 7 Done Sent Done false
31921 9455 Acked Read 9 Done Sent Done false
31922 9456 Acked Write 6 Done Sent Done false
31923 9457 Acked Read 9 Done Sent Done false
31924 9458 Acked Read 7 Done Sent Done false
31925 9459 Acked Write 1 Done Sent Done false
31926 9460 Acked Read 1 Done Sent Done false
31927 9461 Acked Read 9 Done Sent Done false
31928 9462 Acked Write 10 Done Sent Done false
31929 9463 Acked Flush 0 Done Sent Done false
31930 9464 Acked Write 5 Done Sent Done false
31931 9465 Acked Write 9 Done Sent Done false
31932 9466 Acked Write 6 Done Sent Done false
31933 9467 Acked Flush 0 Done Sent Done false
31934 STATES DS:0 DS:1 DS:2 TOTAL
31935 Sent 0 302 0 302
31936 Done 302 0 302 604
31937 Skipped 0 0 0 0
31938 Error 0 0 0 0
31939 Last Flush: 9467 9165 9467
31940 Downstairs last five completed: 9165 9164 9163 9162 9161
31941 Upstairs last five completed: 9467 9466 9465 9464 9463
31942 CLIENT: Up:0 ds:302 act:2
319432024-11-07T14:09:59.812ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319442024-11-07T14:09:59.812ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60351) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319452024-11-07T14:09:59.813ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319462024-11-07T14:09:59.814ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319472024-11-07T14:09:59.814ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319482024-11-07T14:09:59.814ZINFOcrucible: send last flush ID to this DS: 9165 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319492024-11-07T14:09:59.814ZINFOcrucible: Replied this last flush ID: 9165 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319502024-11-07T14:09:59.814ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319512024-11-07T14:09:59.814ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319522024-11-07T14:09:59.814ZINFOcrucible: [1] client re-new 302 jobs since flush 9165 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319532024-11-07T14:09:59.814ZINFOcrucible: [1] Replayed 302 jobs since flush: 9165 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
31954 ----------------------------------------------------------------
31955 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31956 Downstairs last five completed: 9467 9466 9465 9464 9463
31957 Upstairs last five completed: 9467 9466 9465 9464 9463
31958 CLIENT: Up:0 ds:0 act:3
31959 Replay: All jobs finished
31960 Main task has work to do, go find it
31961 got dsc Stop(2)
31962 stop 2
31963 Nov 07 14:10:02.104 INFO request completed, latency_us: 365, response_code: 204[8830] Got stop action so:false kr:false
31964 , uri: /stop/cid/2, method: POST, req_id: 3adc0d90-fa2c-4aa0-b397-98b99630a3f1, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
319652024-11-07T14:10:02.105ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319662024-11-07T14:10:02.105ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319672024-11-07T14:10:02.105ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319682024-11-07T14:10:02.105ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
319692024-11-07T14:10:02.105ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
31970 Nov 07 14:10:02.105 INFO request completed, latency_us: 124, response_code: 200, uri: /state/cid/2, method: GET, req_id{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:10:02.105526809Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"":"io task","client":"2","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
31971 : 64c57774-1c31-445e-a55e-749af4007933, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
31972 [8830] Exited with: ExitStatus(unix_wait_status(9))
31973 [8830][2] reports Exit
31974 Nov 07 14:10:06.107 INFO request completed, latency_us: 219, response_code: 200, uri: /state/cid/2, method: GET, req_id: c3becb64-dc1e-4f9d-b21c-e69925a3df03, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
31975 001/300 Read block 1655 len 28672
31976 002/300 Write block 2937 len 16384 data: 7 6 8 8
31977 003/300 Write block 4946 len 12288 data: 4 4 5
31978 004/300 Write block 920 len 12288 data: 5 4 4
31979 005/300 Write block 327 len 36864 data: 7 7 7 7 7 7 7 6 6
31980 006/300 Read block 4415 len 4096
31981 007/300 Write block 4706 len 20480 data: 5 5 5 5 5
31982 008/300 Read block 3036 len 8192
31983 009/300 Read block 1457 len 24576
31984 010/300 Read block 4788 len 16384
31985 011/300 Read block 2148 len 32768
31986 012/300 Write block 1734 len 4096 data: 3
31987 013/300 Read block 3985 len 4096
31988 014/300 Flush
31989 015/300 Write block 3982 len 4096 data: 3
31990 016/300 Read block 2595 len 32768
31991 017/300 Write block 723 len 4096 data: 7
31992 018/300 Write block 3307 len 28672 data: 6 7 7 7 7 6 5
31993 019/300 Read block 266 len 12288
31994 020/300 Read block 4662 len 16384
31995 021/300 Write block 4052 len 4096 data: 4
31996 022/300 Read block 1394 len 32768
31997 023/300 Write block 4204 len 28672 data: 7 5 5 5 3 3 4
31998 024/300 Read block 2727 len 20480
31999 025/300 Flush
32000 026/300 Write block 4416 len 24576 data: 5 7 5 5 5 6
32001 027/300 Flush
32002 028/300 Read block 1428 len 24576
32003 029/300 Write block 1610 len 36864 data: 3 5 4 6 6 6 5 6 5
32004 030/300 Read block 4371 len 32768
32005 031/300 Read block 1960 len 12288
32006 032/300 Read block 588 len 36864
32007 033/300 Write block 1482 len 24576 data: 6 7 8 9 8 6
32008 034/300 Write block 1749 len 32768 data: 8 11 12 10 10 10 8 7
32009 035/300 Read block 3692 len 20480
32010 036/300 Read block 3607 len 28672
32011 037/300 Read block 2259 len 8192
32012 038/300 Write block 3262 len 4096 data: 6
32013 039/300 Read block 3803 len 12288
32014 040/300 Write block 4065 len 16384 data: 6 5 5 5
32015 041/300 Flush
32016 042/300 Read block 3470 len 8192
32017 043/300 Write block 4313 len 16384 data: 9 10 8 7
32018 044/300 Write block 3052 len 24576 data: 7 7 8 5 7 5
32019 045/300 Write block 3541 len 24576 data: 4 5 6 6 9 9
32020 046/300 Write block 4367 len 12288 data: 9 9 9
32021 047/300 Read block 4730 len 8192
32022 048/300 Read block 2423 len 36864
32023 049/300 Read block 1002 len 12288
32024 050/300 Read block 2489 len 20480
32025 051/300 Read block 1736 len 16384
32026 052/300 Read block 4933 len 36864
32027 053/300 Write block 4304 len 16384 data: 4 4 6 7
32028 054/300 Write block 510 len 24576 data: 5 5 6 6 7 8
32029 055/300 Read block 2695 len 28672
32030 056/300 Write block 1169 len 36864 data: 5 5 6 7 7 7 6 6 5
32031 057/300 Write block 2717 len 16384 data: 5 4 4 6
32032 058/300 Flush
32033 059/300 Write block 633 len 16384 data: 5 4 6 7
32034 060/300 Write block 4565 len 8192 data: 5 2
32035 061/300 Write block 2747 len 8192 data: 4 5
32036 062/300 Write block 3312 len 20480 data: 7 6 4 3 3
32037 063/300 Write block 2002 len 28672 data: 6 4 2 3 4 8 9
32038 064/300 Write block 4614 len 12288 data: 5 5 4
32039 065/300 Read block 229 len 28672
32040 066/300 Write block 2649 len 36864 data: 8 8 7 6 5 5 5 6 5
32041 067/300 Write block 1051 len 20480 data: 10 9 8 7 7
32042 068/300 Read block 1544 len 8192
32043 069/300 Write block 1916 len 16384 data: 10 8 8 10
32044 070/300 Read block 1350 len 8192
32045 071/300 Write block 4073 len 36864 data: 4 4 5 6 6 6 4 4 5
32046 072/300 Read block 294 len 20480
32047 073/300 Read block 3893 len 20480
32048 074/300 Write block 3756 len 4096 data: 6
32049 075/300 Read block 2416 len 36864
32050 076/300 Write block 4225 len 24576 data: 7 7 7 9 9 8
32051 077/300 Read block 2312 len 32768
32052 078/300 Flush
32053 079/300 Write block 625 len 32768 data: 7 8 8 7 6 6 7 7
32054 080/300 Read block 3356 len 28672
32055 081/300 Flush
32056 082/300 Write block 1409 len 32768 data: 6 6 4 5 5 4 4 4
32057 083/300 Read block 3162 len 40960
32058 084/300 Read block 2888 len 20480
32059 085/300 Read block 4103 len 16384
32060 086/300 Write block 857 len 24576 data: 8 7 8 9 8 7
32061 087/300 Write block 1789 len 4096 data: 5
32062 088/300 Write block 3594 len 4096 data: 3
32063 089/300 Read block 4431 len 32768
32064 090/300 Read block 1916 len 4096
32065 091/300 Write block 4655 len 36864 data: 5 4 3 5 5 5 7 7 6
32066 092/300 Read block 2609 len 8192
32067 093/300 Read block 4214 len 24576
32068 094/300 Flush
32069 095/300 Flush
32070 096/300 Write block 3179 len 20480 data: 5 4 5 4 3
32071 097/300 Write block 1328 len 28672 data: 9 9 10 9 9 12 10
32072 098/300 Read block 251 len 20480
32073 099/300 Read block 3671 len 32768
32074 100/300 Flush
32075 101/300 Write block 4502 len 8192 data: 6 6
32076 102/300 Read block 3772 len 36864
32077 103/300 Flush
32078 104/300 Read block 3823 len 28672
32079 105/300 Write block 624 len 24576 data: 6 8 9 9 8 7
32080 106/300 Write block 1998 len 28672 data: 6 6 6 6 7 5 3
32081 107/300 Flush
32082 108/300 Read block 1478 len 32768
32083 109/300 Read block 849 len 24576
32084 110/300 Read block 787 len 4096
32085 111/300 Write block 958 len 40960 data: 6 6 4 4 4 4 4 3 2 3
32086 112/300 Read block 3618 len 8192
32087 113/300 Read block 1990 len 20480
32088 114/300 Read block 3374 len 4096
32089 115/300 Write block 2610 len 40960 data: 7 8 8 7 7 7 6 8 7 7
32090 116/300 Write block 2891 len 32768 data: 6 6 5 5 5 3 2 4
32091 117/300 Read block 819 len 16384
32092 118/300 Read block 3706 len 8192
32093 119/300 Read block 280 len 24576
32094 120/300 Write block 2503 len 32768 data: 4 5 3 3 3 4 4 4
32095 121/300 Write block 3372 len 40960 data: 3 6 6 8 10 11 7 7 7 8
32096 122/300 Write block 3140 len 28672 data: 8 6 7 7 5 5 5
32097 123/300 Read block 3806 len 24576
32098 124/300 Write block 4452 len 28672 data: 6 6 4 4 3 3 3
32099 125/300 Flush
32100 126/300 Write block 4172 len 20480 data: 7 9 10 8 7
32101 127/300 Write block 2100 len 20480 data: 3 6 6 4 7
32102 128/300 Read block 4426 len 40960
32103 129/300 Read block 2887 len 12288
32104 130/300 Write block 3246 len 20480 data: 6 5 5 4 3
32105 131/300 Flush
32106 132/300 Write block 3907 len 32768 data: 4 4 4 4 5 5 4 4
32107 133/300 Write block 4587 len 24576 data: 7 8 7 8 9 9
32108 134/300 Write block 4600 len 12288 data: 7 7 7
32109 135/300 Read block 4305 len 40960
32110 136/300 Write block 909 len 20480 data: 5 4 5 6 6
32111 137/300 Read block 4035 len 16384
32112 138/300 Read block 2262 len 32768
32113 139/300 Read block 4601 len 24576
32114 140/300 Write block 3777 len 40960 data: 5 5 4 4 4 4 4 6 5 6
32115 141/300 Read block 1895 len 40960
32116 142/300 Read block 4259 len 24576
32117 143/300 Read block 3601 len 12288
32118 144/300 Read block 2234 len 24576
32119 145/300 Read block 2266 len 20480
32120 146/300 Read block 3040 len 4096
32121 147/300 Read block 3210 len 40960
32122 148/300 Read block 3173 len 36864
32123 149/300 Write block 3528 len 32768 data: 4 4 5 6 6 8 7 5
32124 150/300 Write block 2627 len 36864 data: 4 3 4 4 7 6 6 3 5
32125 151/300 Read block 2633 len 12288
32126 152/300 Write block 4451 len 16384 data: 5 7 7 5
32127 153/300 Flush
32128 154/300 Read block 4172 len 4096
32129 155/300 Flush
32130 156/300 Write block 2219 len 40960 data: 4 4 3 4 3 4 4 6 6 7
32131 157/300 Write block 3540 len 24576 data: 3 5 6 7 7 10
32132 158/300 Read block 2541 len 4096
32133 159/300 Write block 2723 len 20480 data: 7 8 8 8 8
32134 160/300 Flush
32135 161/300 Read block 1922 len 40960
32136 162/300 Write block 194 len 16384 data: 5 6 5 5
32137 163/300 Flush
32138 164/300 Write block 4252 len 36864 data: 6 6 6 6 6 6 7 6 7
32139 165/300 Read block 4543 len 40960
32140 166/300 Write block 3979 len 36864 data: 5 3 4 4 4 3 6 6 6
32141 167/300 Read block 3366 len 16384
32142 168/300 Write block 1494 len 20480 data: 9 7 7 6 6
32143 169/300 Read block 4226 len 24576
32144 170/300 Write block 3082 len 36864 data: 8 10 11 11 9 7 8 8 7
32145 171/300 Write block 2759 len 24576 data: 4 3 4 5 4 3
32146 172/300 Read block 4441 len 36864
32147 173/300 Read block 3223 len 16384
32148 174/300 Write block 4647 len 32768 data: 10 11 9 7 7 6 4 3
32149 175/300 Write block 3545 len 16384 data: 11 10 8 10
32150 176/300 Read block 3399 len 36864
32151 177/300 Read block 2228 len 4096
32152 178/300 Flush
32153 179/300 Read block 3487 len 16384
32154 180/300 Read block 1683 len 36864
32155 181/300 Read block 1040 len 16384
32156 182/300 Read block 2665 len 40960
32157 183/300 Write block 5 len 40960 data: 7 7 6 6 5 5 5 6 6 5
32158 184/300 Flush
32159 185/300 Write block 439 len 16384 data: 8 9 7 9
32160 186/300 Read block 213 len 32768
32161 187/300 Write block 823 len 36864 data: 5 6 6 5 4 7 8 10 8
32162 188/300 Read block 4418 len 36864
32163 189/300 Write block 4503 len 40960 data: 7 7 5 5 6 6 5 5 5 6
32164 190/300 Write block 1098 len 8192 data: 5 5
32165 191/300 Write block 1776 len 20480 data: 6 6 8 8 8
32166 192/300 Read block 1948 len 12288
32167 193/300 Write block 1099 len 20480 data: 6 5 5 4 4
32168 194/300 Read block 163 len 32768
32169 195/300 Flush
32170 196/300 Read block 1887 len 28672
32171 197/300 Read block 408 len 40960
32172 198/300 Read block 3887 len 32768
32173 199/300 Write block 3953 len 40960 data: 5 5 6 7 6 6 6 4 3 4
32174 200/300 Write block 1711 len 20480 data: 10 8 7 8 7
32175 201/300 Write block 4784 len 32768 data: 5 6 6 7 8 8 6 5
32176 202/300 Read block 3742 len 16384
32177 203/300 Read block 699 len 4096
32178 204/300 Write block 1537 len 36864 data: 4 4 5 5 5 5 5 5 3
32179 205/300 Read block 4688 len 36864
32180 206/300 Write block 3998 len 28672 data: 5 6 7 7 7 7 6
32181 207/300 Flush
32182 208/300 Write block 3534 len 12288 data: 8 6 6
32183 209/300 Write block 1944 len 20480 data: 10 9 10 7 8
32184 210/300 Read block 4554 len 20480
32185 211/300 Read block 351 len 20480
32186 212/300 Read block 2497 len 8192
32187 213/300 Read block 3231 len 40960
32188 214/300 Read block 4587 len 4096
32189 215/300 Flush
32190 216/300 Read block 478 len 40960
32191 217/300 Read block 1460 len 28672
32192 218/300 Read block 1992 len 12288
32193 219/300 Write block 1821 len 20480 data: 4 5 7 8 7
32194 220/300 Write block 2652 len 12288 data: 7 6 6
32195 221/300 Write block 3308 len 12288 data: 8 8 8
32196 222/300 Read block 404 len 8192
32197 223/300 Write block 3187 len 24576 data: 5 4 4 3 4 4
32198 224/300 Write block 1575 len 24576 data: 8 8 6 4 5 7
32199 225/300 Read block 3824 len 16384
32200 226/300 Write block 3226 len 24576 data: 5 6 6 5 6 5
32201 227/300 Write block 579 len 8192 data: 4 4
32202 228/300 Read block 652 len 40960
32203 229/300 Flush
32204 230/300 Flush
32205 231/300 Read block 1993 len 8192
32206 232/300 Write block 19 len 32768 data: 6 7 7 6 6 7 9 11
32207 233/300 Read block 4732 len 40960
32208 234/300 Read block 1270 len 24576
32209 235/300 Read block 497 len 40960
32210 236/300 Write block 4534 len 36864 data: 10 7 6 6 7 5 4 5 6
32211 237/300 Write block 4278 len 36864 data: 6 5 6 6 4 4 4 4 5
32212 238/300 Flush
32213 239/300 Write block 4178 len 24576 data: 9 8 7 8 7 7
32214 240/300 Read block 1073 len 12288
32215 241/300 Write block 3464 len 24576 data: 7 7 5 5 5 5
32216 242/300 Read block 4918 len 36864
32217 243/300 Write block 3811 len 12288 data: 4 4 7
32218 244/300 Write block 948 len 4096 data: 3
32219 245/300 Write block 1488 len 36864 data: 6 6 6 6 6 8 10 8 8
32220 246/300 Flush
32221 247/300 Read block 1989 len 8192
32222 248/300 Write block 3626 len 16384 data: 6 6 7 8
32223 249/300 Write block 4425 len 24576 data: 3 4 3 3 3 2
32224 250/300 Write block 1747 len 4096 data: 4
32225 251/300 Write block 4264 len 40960 data: 5 5 6 4 5 7 7 10 10 9
32226 252/300 Read block 1943 len 20480
32227 253/300 Write block 2913 len 28672 data: 9 8 7 7 7 7 7
32228 254/300 Write block 4118 len 36864 data: 6 8 7 8 9 8 9 7 7
32229 255/300 Read block 4870 len 24576
32230 256/300 Read block 3265 len 24576
32231 257/300 Write block 2204 len 16384 data: 4 5 6 6
32232 258/300 Write block 2730 len 4096 data: 7
32233 259/300 Write block 1759 len 28672 data: 6 5 4 6 5 7 8
32234 260/300 Write block 3074 len 28672 data: 5 5 7 5 4 4 7
32235 261/300 Write block 185 len 28672 data: 7 9 8 8 8 8 6
32236 262/300 Read block 4067 len 4096
32237 263/300 Flush
32238 264/300 Write block 3351 len 16384 data: 3 5 5 5
32239 265/300 Read block 620 len 40960
32240 266/300 Write block 1853 len 4096 data: 9
32241 267/300 Write block 1167 len 28672 data: 5 6 6 6 7 8 8
32242 268/300 Read block 2298 len 12288
32243 269/300 Write block 2878 len 12288 data: 3 4 6
32244 270/300 Read block 2182 len 40960
32245 271/300 Read block 3355 len 16384
32246 272/300 Read block 1506 len 28672
32247 273/300 Write block 2851 len 8192 data: 8 9
32248 274/300 Flush
32249 275/300 Write block 1656 len 24576 data: 7 8 6 8 9 9
32250 276/300 Flush
32251 277/300 Read block 3186 len 28672
32252 278/300 Read block 647 len 16384
32253 279/300 Read block 4834 len 4096
32254 280/300 Read block 2132 len 40960
32255 281/300 Write block 1395 len 40960 data: 5 8 9 9 9 9 8 7 8 8
32256 282/300 Read block 5 len 40960
32257 283/300 Read block 578 len 16384
32258 284/300 Read block 1725 len 12288
32259 285/300 Write block 3070 len 4096 data: 3
32260 286/300 Write block 2029 len 8192 data: 7 6
32261 287/300 Write block 3132 len 32768 data: 7 8 10 11 11 9 8 7
32262 288/300 Read block 541 len 12288
32263 289/300 Read block 1540 len 28672
32264 290/300 Read block 1000 len 8192
32265 291/300 Write block 2598 len 32768 data: 7 8 6 6 7 7 7 6
32266 292/300 Read block 3400 len 24576
32267 293/300 Write block 2601 len 32768 data: 7 8 8 8 7 6 6 5
32268 294/300 Read block 2149 len 20480
32269 295/300 Read block 2768 len 36864
32270 296/300 Write block 4982 len 40960 data: 5 5 5 5 5 6 7 8 8 7
32271 297/300 Write block 4844 len 24576 data: 8 8 8 8 6 5
32272 298/300 Read block 1191 len 28672
32273 299/300 Write block 1759 len 16384 data: 7 6 5 7
32274 300/300 Flush
32275 301/300 Flush
32276 Main task has work to do, go find it
32277 got dsc Start(2)
32278 start 2
32279 Nov 07 14:10:06.406 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/2, method: POST, req_id: 8b489f62-9b10-4571-9f29-8655912172c7, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
32280 Replay: started 2, returned:Ok(())
32281 ----------------------------------------------------------------
32282 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32283 States: Active Active Offline
32284 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32285 9468 Acked Read 7 Done Done Sent false
32286 9469 Acked Write 4 Done Done Sent false
32287 9470 Acked Write 3 Done Done Sent false
32288 9471 Acked Write 3 Done Done Sent false
32289 9472 Acked Write 9 Done Done Sent false
32290 9473 Acked Read 1 Done Done Sent false
32291 9474 Acked Write 5 Done Done Sent false
32292 9475 Acked Read 2 Done Done Sent false
32293 9476 Acked Read 6 Done Done Sent false
32294 9477 Acked Read 4 Done Done Sent false
32295 9478 Acked Read 8 Done Done Sent false
32296 9479 Acked Write 1 Done Done Sent false
32297 9480 Acked Read 1 Done Done Sent false
32298 9481 Acked Flush 0 Done Done Sent false
32299 9482 Acked Write 1 Done Done Sent false
32300 9483 Acked Read 8 Done Done Sent false
32301 9484 Acked Write 1 Done Done Sent false
32302 9485 Acked Write 7 Done Done Sent false
32303 9486 Acked Read 3 Done Done Sent false
32304 9487 Acked Read 4 Done Done Sent false
32305 9488 Acked Write 1 Done Done Sent false
32306 9489 Acked Read 8 Done Done Sent false
32307 9490 Acked Write 7 Done Done Sent false
32308 9491 Acked Read 5 Done Done Sent false
32309 9492 Acked Flush 0 Done Done Sent false
32310 9493 Acked Write 6 Done Done Sent false
32311 9494 Acked Flush 0 Done Done Sent false
32312 9495 Acked Read 6 Done Done Sent false
32313 9496 Acked Write 9 Done Done Sent false
32314 9497 Acked Read 8 Done Done Sent false
32315 9498 Acked Read 3 Done Done Sent false
32316 9499 Acked Read 9 Done Done Sent false
32317 9500 Acked Write 6 Done Done Sent false
32318 9501 Acked Write 8 Done Done Sent false
32319 9502 Acked Read 5 Done Done Sent false
32320 9503 Acked Read 7 Done Done Sent false
32321 9504 Acked Read 2 Done Done Sent false
32322 9505 Acked Write 1 Done Done Sent false
32323 9506 Acked Read 3 Done Done Sent false
32324 9507 Acked Write 4 Done Done Sent false
32325 9508 Acked Flush 0 Done Done Sent false
32326 9509 Acked Read 2 Done Done Sent false
32327 9510 Acked Write 4 Done Done Sent false
32328 9511 Acked Write 6 Done Done Sent false
32329 9512 Acked Write 6 Done Done Sent false
32330 9513 Acked Write 3 Done Done Sent false
32331 9514 Acked Read 2 Done Done Sent false
32332 9515 Acked Read 9 Done Done Sent false
32333 9516 Acked Read 3 Done Done Sent false
32334 9517 Acked Read 5 Done Done Sent false
32335 9518 Acked Read 4 Done Done Sent false
32336 9519 Acked Read 9 Done Done Sent false
32337 9520 Acked Write 4 Done Done Sent false
32338 9521 Acked Write 6 Done Done Sent false
32339 9522 Acked Read 7 Done Done Sent false
32340 9523 Acked Write 9 Done Done Sent false
32341 9524 Acked Write 4 Done Done Sent false
32342 9525 Acked Flush 0 Done Done Sent false
32343 9526 Acked Write 4 Done Done Sent false
32344 9527 Acked Write 2 Done Done Sent false
32345 9528 Acked Write 2 Done Done Sent false
32346 9529 Acked Write 5 Done Done Sent false
32347 9530 Acked Write 7 Done Done Sent false
32348 9531 Acked Write 3 Done Done Sent false
32349 9532 Acked Read 7 Done Done Sent false
32350 9533 Acked Write 9 Done Done Sent false
32351 9534 Acked Write 5 Done Done Sent false
32352 9535 Acked Read 2 Done Done Sent false
32353 9536 Acked Write 4 Done Done Sent false
32354 9537 Acked Read 2 Done Done Sent false
32355 9538 Acked Write 9 Done Done Sent false
32356 9539 Acked Read 5 Done Done Sent false
32357 9540 Acked Read 5 Done Done Sent false
32358 9541 Acked Write 1 Done Done Sent false
32359 9542 Acked Read 9 Done Done Sent false
32360 9543 Acked Write 6 Done Done Sent false
32361 9544 Acked Read 8 Done Done Sent false
32362 9545 Acked Flush 0 Done Done Sent false
32363 9546 Acked Write 8 Done Done Sent false
32364 9547 Acked Read 7 Done Done Sent false
32365 9548 Acked Flush 0 Done Done Sent false
32366 9549 Acked Write 8 Done Done Sent false
32367 9550 Acked Read 10 Done Done Sent false
32368 9551 Acked Read 5 Done Done Sent false
32369 9552 Acked Read 4 Done Done Sent false
32370 9553 Acked Write 6 Done Done Sent false
32371 9554 Acked Write 1 Done Done Sent false
32372 9555 Acked Write 1 Done Done Sent false
32373 9556 Acked Read 8 Done Done Sent false
32374 9557 Acked Read 1 Done Done Sent false
32375 9558 Acked Write 9 Done Done Sent false
32376 9559 Acked Read 2 Done Done Sent false
32377 9560 Acked Read 6 Done Done Sent false
32378 9561 Acked Flush 0 Done Done Sent false
32379 9562 Acked Flush 0 Done Done Sent false
32380 9563 Acked Write 5 Done Done Sent false
32381 9564 Acked Write 7 Done Done Sent false
32382 9565 Acked Read 5 Done Done Sent false
32383 9566 Acked Read 8 Done Done Sent false
32384 9567 Acked Flush 0 Done Done Sent false
32385 9568 Acked Write 2 Done Done Sent false
32386 9569 Acked Read 9 Done Done Sent false
32387 9570 Acked Flush 0 Done Done Sent false
32388 9571 Acked Read 7 Done Done Sent false
32389 9572 Acked Write 6 Done Done Sent false
32390 9573 Acked Write 7 Done Done Sent false
32391 9574 Acked Flush 0 Done Done Sent false
32392 9575 Acked Read 8 Done Done Sent false
32393 9576 Acked Read 6 Done Done Sent false
32394 9577 Acked Read 1 Done Done Sent false
32395 9578 Acked Write 10 Done Done Sent false
32396 9579 Acked Read 2 Done Done Sent false
32397 9580 Acked Read 5 Done Done Sent false
32398 9581 Acked Read 1 Done Done Sent false
32399 9582 Acked Write 10 Done Done Sent false
32400 9583 Acked Write 8 Done Done Sent false
32401 9584 Acked Read 4 Done Done Sent false
32402 9585 Acked Read 2 Done Done Sent false
32403 9586 Acked Read 6 Done Done Sent false
32404 9587 Acked Write 8 Done Done Sent false
32405 9588 Acked Write 10 Done Done Sent false
32406 9589 Acked Write 7 Done Done Sent false
32407 9590 Acked Read 6 Done Done Sent false
32408 9591 Acked Write 7 Done Done Sent false
32409 9592 Acked Flush 0 Done Done Sent false
32410 9593 Acked Write 5 Done Done Sent false
32411 9594 Acked Write 5 Done Done Sent false
32412 9595 Acked Read 10 Done Done Sent false
32413 9596 Acked Read 3 Done Done Sent false
32414 9597 Acked Write 5 Done Done Sent false
32415 9598 Acked Flush 0 Done Done Sent false
32416 9599 Acked Write 8 Done Done Sent false
32417 9600 Acked Write 6 Done Done Sent false
32418 9601 Acked Write 3 Done Done Sent false
32419 9602 Acked Read 10 Done Done Sent false
32420 9603 Acked Write 5 Done Done Sent false
32421 9604 Acked Read 4 Done Done Sent false
32422 9605 Acked Read 8 Done Done Sent false
32423 9606 Acked Read 6 Done Done Sent false
32424 9607 Acked Write 10 Done Done Sent false
32425 9608 Acked Read 10 Done Done Sent false
32426 9609 Acked Read 6 Done Done Sent false
32427 9610 Acked Read 3 Done Done Sent false
32428 9611 Acked Read 6 Done Done Sent false
32429 9612 Acked Read 5 Done Done Sent false
32430 9613 Acked Read 1 Done Done Sent false
32431 9614 Acked Read 10 Done Done Sent false
32432 9615 Acked Read 9 Done Done Sent false
32433 9616 Acked Write 8 Done Done Sent false
32434 9617 Acked Write 9 Done Done Sent false
32435 9618 Acked Read 3 Done Done Sent false
32436 9619 Acked Write 4 Done Done Sent false
32437 9620 Acked Flush 0 Done Done Sent false
32438 9621 Acked Read 1 Done Done Sent false
32439 9622 Acked Flush 0 Done Done Sent false
32440 9623 Acked Write 10 Done Done Sent false
32441 9624 Acked Write 6 Done Done Sent false
32442 9625 Acked Read 1 Done Done Sent false
32443 9626 Acked Write 5 Done Done Sent false
32444 9627 Acked Flush 0 Done Done Sent false
32445 9628 Acked Read 10 Done Done Sent false
32446 9629 Acked Write 4 Done Done Sent false
32447 9630 Acked Flush 0 Done Done Sent false
32448 9631 Acked Write 9 Done Done Sent false
32449 9632 Acked Read 10 Done Done Sent false
32450 9633 Acked Write 9 Done Done Sent false
32451 9634 Acked Read 4 Done Done Sent false
32452 9635 Acked Write 5 Done Done Sent false
32453 9636 Acked Read 6 Done Done Sent false
32454 9637 Acked Write 9 Done Done Sent false
32455 9638 Acked Write 6 Done Done Sent false
32456 9639 Acked Read 9 Done Done Sent false
32457 9640 Acked Read 4 Done Done Sent false
32458 9641 Acked Write 8 Done Done Sent false
32459 9642 Acked Write 4 Done Done Sent false
32460 9643 Acked Read 9 Done Done Sent false
32461 9644 Acked Read 1 Done Done Sent false
32462 9645 Acked Flush 0 Done Done Sent false
32463 9646 Acked Read 4 Done Done Sent false
32464 9647 Acked Read 9 Done Done Sent false
32465 9648 Acked Read 4 Done Done Sent false
32466 9649 Acked Read 10 Done Done Sent false
32467 9650 Acked Write 10 Done Done Sent false
32468 9651 Acked Flush 0 Done Done Sent false
32469 9652 Acked Write 4 Done Done Sent false
32470 9653 Acked Read 8 Done Done Sent false
32471 9654 Acked Write 9 Done Done Sent false
32472 9655 Acked Read 9 Done Done Sent false
32473 9656 Acked Write 10 Done Done Sent false
32474 9657 Acked Write 2 Done Done Sent false
32475 9658 Acked Write 5 Done Done Sent false
32476 9659 Acked Read 3 Done Done Sent false
32477 9660 Acked Write 5 Done Done Sent false
32478 9661 Acked Read 8 Done Done Sent false
32479 9662 Acked Flush 0 Done Done Sent false
32480 9663 Acked Read 7 Done Done Sent false
32481 9664 Acked Read 10 Done Done Sent false
32482 9665 Acked Read 8 Done Done Sent false
32483 9666 Acked Write 10 Done Done Sent false
32484 9667 Acked Write 5 Done Done Sent false
32485 9668 Acked Write 8 Done Done Sent false
32486 9669 Acked Read 4 Done Done Sent false
32487 9670 Acked Read 1 Done Done Sent false
32488 9671 Acked Write 9 Done Done Sent false
32489 9672 Acked Read 9 Done Done Sent false
32490 9673 Acked Write 7 Done Done Sent false
32491 9674 Acked Flush 0 Done Done Sent false
32492 9675 Acked Write 3 Done Done Sent false
32493 9676 Acked Write 5 Done Done Sent false
32494 9677 Acked Read 5 Done Done Sent false
32495 9678 Acked Read 5 Done Done Sent false
32496 9679 Acked Read 2 Done Done Sent false
32497 9680 Acked Read 10 Done Done Sent false
32498 9681 Acked Read 1 Done Done Sent false
32499 9682 Acked Flush 0 Done Done Sent false
32500 9683 Acked Read 10 Done Done Sent false
32501 9684 Acked Read 7 Done Done Sent false
32502 9685 Acked Read 3 Done Done Sent false
32503 9686 Acked Write 5 Done Done Sent false
32504 9687 Acked Write 3 Done Done Sent false
32505 9688 Acked Write 3 Done Done Sent false
32506 9689 Acked Read 2 Done Done Sent false
32507 9690 Acked Write 6 Done Done Sent false
32508 9691 Acked Write 6 Done Done Sent false
32509 9692 Acked Read 4 Done Done Sent false
32510 9693 Acked Write 6 Done Done Sent false
32511 9694 Acked Write 2 Done Done Sent false
32512 9695 Acked Read 10 Done Done Sent false
32513 9696 Acked Flush 0 Done Done Sent false
32514 9697 Acked Flush 0 Done Done Sent false
32515 9698 Acked Read 2 Done Done Sent false
32516 9699 Acked Write 8 Done Done Sent false
32517 9700 Acked Read 10 Done Done Sent false
32518 9701 Acked Read 6 Done Done Sent false
32519 9702 Acked Read 10 Done Done Sent false
32520 9703 Acked Write 9 Done Done Sent false
32521 9704 Acked Write 9 Done Done Sent false
32522 9705 Acked Flush 0 Done Done Sent false
32523 9706 Acked Write 6 Done Done Sent false
32524 9707 Acked Read 3 Done Done Sent false
32525 9708 Acked Write 6 Done Done Sent false
32526 9709 Acked Read 9 Done Done Sent false
32527 9710 Acked Write 3 Done Done Sent false
32528 9711 Acked Write 1 Done Done Sent false
32529 9712 Acked Write 9 Done Done Sent false
32530 9713 Acked Flush 0 Done Done Sent false
32531 9714 Acked Read 2 Done Done Sent false
32532 9715 Acked Write 4 Done Done Sent false
32533 9716 Acked Write 6 Done Done Sent false
32534 9717 Acked Write 1 Done Done Sent false
32535 9718 Acked Write 10 Done Done Sent false
32536 9719 Acked Read 5 Done Done Sent false
32537 9720 Acked Write 7 Done Done Sent false
32538 9721 Acked Write 9 Done Done Sent false
32539 9722 Acked Read 6 Done Done Sent false
32540 9723 Acked Read 6 Done Done Sent false
32541 9724 Acked Write 4 Done Done Sent false
32542 9725 Acked Write 1 Done Done Sent false
32543 9726 Acked Write 7 Done Done Sent false
32544 9727 Acked Write 7 Done Done Sent false
32545 9728 Acked Write 7 Done Done Sent false
32546 9729 Acked Read 1 Done Done Sent false
32547 9730 Acked Flush 0 Done Done Sent false
32548 9731 Acked Write 4 Done Done Sent false
32549 9732 Acked Read 10 Done Done Sent false
32550 9733 Acked Write 1 Done Done Sent false
32551 9734 Acked Write 7 Done Done Sent false
32552 9735 Acked Read 3 Done Done Sent false
32553 9736 Acked Write 3 Done Done Sent false
32554 9737 Acked Read 10 Done Done Sent false
32555 9738 Acked Read 4 Done Done Sent false
32556 9739 Acked Read 7 Done Done Sent false
32557 9740 Acked Write 2 Done Done Sent false
32558 9741 Acked Flush 0 Done Done Sent false
32559 9742 Acked Write 6 Done Done Sent false
32560 9743 Acked Flush 0 Done Done Sent false
32561 9744 Acked Read 7 Done Done Sent false
32562 9745 Acked Read 4 Done Done Sent false
32563 9746 Acked Read 1 Done Done Sent false
32564 9747 Acked Read 10 Done Done Sent false
32565 9748 Acked Write 10 Done Done Sent false
32566 9749 Acked Read 10 Done Done Sent false
32567 9750 Acked Read 4 Done Done Sent false
32568 9751 Acked Read 3 Done Done Sent false
32569 9752 Acked Write 1 Done Done Sent false
32570 9753 Acked Write 2 Done Done Sent false
32571 9754 Acked Write 8 Done Done Sent false
32572 9755 Acked Read 3 Done Done Sent false
32573 9756 Acked Read 7 Done Done Sent false
32574 9757 Acked Read 2 Done Done Sent false
32575 9758 Acked Write 8 Done Done Sent false
32576 9759 Acked Read 6 Done Done Sent false
32577 9760 Acked Write 8 Done Done Sent false
32578 9761 Acked Read 5 Done Done Sent false
32579 9762 Acked Read 9 Done Done Sent false
32580 9763 Acked Write 10 Done Done Sent false
32581 9764 Acked Write 6 Done Done Sent false
32582 9765 Acked Read 7 Done Done Sent false
32583 9766 Acked Write 4 Done Done Sent false
32584 9767 Acked Flush 0 Done Done Sent false
32585 9768 Acked Flush 0 Done Done Sent false
32586 STATES DS:0 DS:1 DS:2 TOTAL
32587 Sent 0 0 301 301
32588 Done 301 301 0 602
32589 Skipped 0 0 0 0
32590 Error 0 0 0 0
32591 Last Flush: 9768 9768 9467
32592 Downstairs last five completed: 9467 9466 9465 9464 9463
32593 Upstairs last five completed: 9768 9767 9766 9765 9764
32594 CLIENT: Up:0 ds:301 act:2
32595 ----------------------------------------------------------------
32596 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32597 States: Active Active Offline
32598 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32599 9468 Acked Read 7 Done Done Sent false
32600 9469 Acked Write 4 Done Done Sent false
32601 9470 Acked Write 3 Done Done Sent false
32602 9471 Acked Write 3 Done Done Sent false
32603 9472 Acked Write 9 Done Done Sent false
32604 9473 Acked Read 1 Done Done Sent false
32605 9474 Acked Write 5 Done Done Sent false
32606 9475 Acked Read 2 Done Done Sent false
32607 9476 Acked Read 6 Done Done Sent false
32608 9477 Acked Read 4 Done Done Sent false
32609 9478 Acked Read 8 Done Done Sent false
32610 9479 Acked Write 1 Done Done Sent false
32611 9480 Acked Read 1 Done Done Sent false
32612 9481 Acked Flush 0 Done Done Sent false
32613 9482 Acked Write 1 Done Done Sent false
32614 9483 Acked Read 8 Done Done Sent false
32615 9484 Acked Write 1 Done Done Sent false
32616 9485 Acked Write 7 Done Done Sent false
32617 9486 Acked Read 3 Done Done Sent false
32618 9487 Acked Read 4 Done Done Sent false
32619 9488 Acked Write 1 Done Done Sent false
32620 9489 Acked Read 8 Done Done Sent false
32621 9490 Acked Write 7 Done Done Sent false
32622 9491 Acked Read 5 Done Done Sent false
32623 9492 Acked Flush 0 Done Done Sent false
32624 9493 Acked Write 6 Done Done Sent false
32625 9494 Acked Flush 0 Done Done Sent false
32626 9495 Acked Read 6 Done Done Sent false
32627 9496 Acked Write 9 Done Done Sent false
32628 9497 Acked Read 8 Done Done Sent false
32629 9498 Acked Read 3 Done Done Sent false
32630 9499 Acked Read 9 Done Done Sent false
32631 9500 Acked Write 6 Done Done Sent false
32632 9501 Acked Write 8 Done Done Sent false
32633 9502 Acked Read 5 Done Done Sent false
32634 9503 Acked Read 7 Done Done Sent false
32635 9504 Acked Read 2 Done Done Sent false
32636 9505 Acked Write 1 Done Done Sent false
32637 9506 Acked Read 3 Done Done Sent false
32638 9507 Acked Write 4 Done Done Sent false
32639 9508 Acked Flush 0 Done Done Sent false
32640 9509 Acked Read 2 Done Done Sent false
32641 9510 Acked Write 4 Done Done Sent false
32642 9511 Acked Write 6 Done Done Sent false
32643 9512 Acked Write 6 Done Done Sent false
32644 9513 Acked Write 3 Done Done Sent false
32645 9514 Acked Read 2 Done Done Sent false
32646 9515 Acked Read 9 Done Done Sent false
32647 9516 Acked Read 3 Done Done Sent false
32648 9517 Acked Read 5 Done Done Sent false
32649 9518 Acked Read 4 Done Done Sent false
32650 9519 Acked Read 9 Done Done Sent false
32651 9520 Acked Write 4 Done Done Sent false
32652 9521 Acked Write 6 Done Done Sent false
32653 9522 Acked Read 7 Done Done Sent false
32654 9523 Acked Write 9 Done Done Sent false
32655 9524 Acked Write 4 Done Done Sent false
32656 9525 Acked Flush 0 Done Done Sent false
32657 9526 Acked Write 4 Done Done Sent false
32658 9527 Acked Write 2 Done Done Sent false
32659 9528 Acked Write 2 Done Done Sent false
32660 9529 Acked Write 5 Done Done Sent false
32661 9530 Acked Write 7 Done Done Sent false
32662 9531 Acked Write 3 Done Done Sent false
32663 9532 Acked Read 7 Done Done Sent false
32664 9533 Acked Write 9 Done Done Sent false
32665 9534 Acked Write 5 Done Done Sent false
32666 9535 Acked Read 2 Done Done Sent false
32667 9536 Acked Write 4 Done Done Sent false
32668 9537 Acked Read 2 Done Done Sent false
32669 9538 Acked Write 9 Done Done Sent false
32670 9539 Acked Read 5 Done Done Sent false
32671 9540 Acked Read 5 Done Done Sent false
32672 9541 Acked Write 1 Done Done Sent false
32673 9542 Acked Read 9 Done Done Sent false
32674 9543 Acked Write 6 Done Done Sent false
32675 9544 Acked Read 8 Done Done Sent false
32676 9545 Acked Flush 0 Done Done Sent false
32677 9546 Acked Write 8 Done Done Sent false
32678 9547 Acked Read 7 Done Done Sent false
32679 9548 Acked Flush 0 Done Done Sent false
32680 9549 Acked Write 8 Done Done Sent false
32681 9550 Acked Read 10 Done Done Sent false
32682 9551 Acked Read 5 Done Done Sent false
32683 9552 Acked Read 4 Done Done Sent false
32684 9553 Acked Write 6 Done Done Sent false
32685 9554 Acked Write 1 Done Done Sent false
32686 9555 Acked Write 1 Done Done Sent false
32687 9556 Acked Read 8 Done Done Sent false
32688 9557 Acked Read 1 Done Done Sent false
32689 9558 Acked Write 9 Done Done Sent false
32690 9559 Acked Read 2 Done Done Sent false
32691 9560 Acked Read 6 Done Done Sent false
32692 9561 Acked Flush 0 Done Done Sent false
32693 9562 Acked Flush 0 Done Done Sent false
32694 9563 Acked Write 5 Done Done Sent false
32695 9564 Acked Write 7 Done Done Sent false
32696 9565 Acked Read 5 Done Done Sent false
32697 9566 Acked Read 8 Done Done Sent false
32698 9567 Acked Flush 0 Done Done Sent false
32699 9568 Acked Write 2 Done Done Sent false
32700 9569 Acked Read 9 Done Done Sent false
32701 9570 Acked Flush 0 Done Done Sent false
32702 9571 Acked Read 7 Done Done Sent false
32703 9572 Acked Write 6 Done Done Sent false
32704 9573 Acked Write 7 Done Done Sent false
32705 9574 Acked Flush 0 Done Done Sent false
32706 9575 Acked Read 8 Done Done Sent false
32707 9576 Acked Read 6 Done Done Sent false
32708 9577 Acked Read 1 Done Done Sent false
32709 9578 Acked Write 10 Done Done Sent false
32710 9579 Acked Read 2 Done Done Sent false
32711 9580 Acked Read 5 Done Done Sent false
32712 9581 Acked Read 1 Done Done Sent false
32713 9582 Acked Write 10 Done Done Sent false
32714 9583 Acked Write 8 Done Done Sent false
32715 9584 Acked Read 4 Done Done Sent false
32716 9585 Acked Read 2 Done Done Sent false
32717 9586 Acked Read 6 Done Done Sent false
32718 9587 Acked Write 8 Done Done Sent false
32719 9588 Acked Write 10 Done Done Sent false
32720 9589 Acked Write 7 Done Done Sent false
32721 9590 Acked Read 6 Done Done Sent false
32722 9591 Acked Write 7 Done Done Sent false
32723 9592 Acked Flush 0 Done Done Sent false
32724 9593 Acked Write 5 Done Done Sent false
32725 9594 Acked Write 5 Done Done Sent false
32726 9595 Acked Read 10 Done Done Sent false
32727 9596 Acked Read 3 Done Done Sent false
32728 9597 Acked Write 5 Done Done Sent false
32729 9598 Acked Flush 0 Done Done Sent false
32730 9599 Acked Write 8 Done Done Sent false
32731 9600 Acked Write 6 Done Done Sent false
32732 9601 Acked Write 3 Done Done Sent false
32733 9602 Acked Read 10 Done Done Sent false
32734 9603 Acked Write 5 Done Done Sent false
32735 9604 Acked Read 4 Done Done Sent false
32736 9605 Acked Read 8 Done Done Sent false
32737 9606 Acked Read 6 Done Done Sent false
32738 9607 Acked Write 10 Done Done Sent false
32739 9608 Acked Read 10 Done Done Sent false
32740 9609 Acked Read 6 Done Done Sent false
32741 9610 Acked Read 3 Done Done Sent false
32742 9611 Acked Read 6 Done Done Sent false
32743 9612 Acked Read 5 Done Done Sent false
32744 9613 Acked Read 1 Done Done Sent false
32745 9614 Acked Read 10 Done Done Sent false
32746 9615 Acked Read 9 Done Done Sent false
32747 9616 Acked Write 8 Done Done Sent false
32748 9617 Acked Write 9 Done Done Sent false
32749 9618 Acked Read 3 Done Done Sent false
32750 9619 Acked Write 4 Done Done Sent false
32751 9620 Acked Flush 0 Done Done Sent false
32752 9621 Acked Read 1 Done Done Sent false
32753 9622 Acked Flush 0 Done Done Sent false
32754 9623 Acked Write 10 Done Done Sent false
32755 9624 Acked Write 6 Done Done Sent false
32756 9625 Acked Read 1 Done Done Sent false
32757 9626 Acked Write 5 Done Done Sent false
32758 9627 Acked Flush 0 Done Done Sent false
32759 9628 Acked Read 10 Done Done Sent false
32760 9629 Acked Write 4 Done Done Sent false
32761 9630 Acked Flush 0 Done Done Sent false
32762 9631 Acked Write 9 Done Done Sent false
32763 9632 Acked Read 10 Done Done Sent false
32764 9633 Acked Write 9 Done Done Sent false
32765 9634 Acked Read 4 Done Done Sent false
32766 9635 Acked Write 5 Done Done Sent false
32767 9636 Acked Read 6 Done Done Sent false
32768 9637 Acked Write 9 Done Done Sent false
32769 9638 Acked Write 6 Done Done Sent false
32770 9639 Acked Read 9 Done Done Sent false
32771 9640 Acked Read 4 Done Done Sent false
32772 9641 Acked Write 8 Done Done Sent false
32773 9642 Acked Write 4 Done Done Sent false
32774 9643 Acked Read 9 Done Done Sent false
32775 9644 Acked Read 1 Done Done Sent false
32776 9645 Acked Flush 0 Done Done Sent false
32777 9646 Acked Read 4 Done Done Sent false
32778 9647 Acked Read 9 Done Done Sent false
32779 9648 Acked Read 4 Done Done Sent false
32780 9649 Acked Read 10 Done Done Sent false
32781 9650 Acked Write 10 Done Done Sent false
32782 9651 Acked Flush 0 Done Done Sent false
32783 9652 Acked Write 4 Done Done Sent false
32784 9653 Acked Read 8 Done Done Sent false
32785 9654 Acked Write 9 Done Done Sent false
32786 9655 Acked Read 9 Done Done Sent false
32787 9656 Acked Write 10 Done Done Sent false
32788 9657 Acked Write 2 Done Done Sent false
32789 9658 Acked Write 5 Done Done Sent false
32790 9659 Acked Read 3 Done Done Sent false
32791 9660 Acked Write 5 Done Done Sent false
32792 9661 Acked Read 8 Done Done Sent false
32793 9662 Acked Flush 0 Done Done Sent false
32794 9663 Acked Read 7 Done Done Sent false
32795 9664 Acked Read 10 Done Done Sent false
32796 9665 Acked Read 8 Done Done Sent false
32797 9666 Acked Write 10 Done Done Sent false
32798 9667 Acked Write 5 Done Done Sent false
32799 9668 Acked Write 8 Done Done Sent false
32800 9669 Acked Read 4 Done Done Sent false
32801 9670 Acked Read 1 Done Done Sent false
32802 9671 Acked Write 9 Done Done Sent false
32803 9672 Acked Read 9 Done Done Sent false
32804 9673 Acked Write 7 Done Done Sent false
32805 9674 Acked Flush 0 Done Done Sent false
32806 9675 Acked Write 3 Done Done Sent false
32807 9676 Acked Write 5 Done Done Sent false
32808 9677 Acked Read 5 Done Done Sent false
32809 9678 Acked Read 5 Done Done Sent false
32810 9679 Acked Read 2 Done Done Sent false
32811 9680 Acked Read 10 Done Done Sent false
32812 9681 Acked Read 1 Done Done Sent false
32813 9682 Acked Flush 0 Done Done Sent false
32814 9683 Acked Read 10 Done Done Sent false
32815 9684 Acked Read 7 Done Done Sent false
32816 9685 Acked Read 3 Done Done Sent false
32817 9686 Acked Write 5 Done Done Sent false
32818 9687 Acked Write 3 Done Done Sent false
32819 9688 Acked Write 3 Done Done Sent false
32820 9689 Acked Read 2 Done Done Sent false
32821 9690 Acked Write 6 Done Done Sent false
32822 9691 Acked Write 6 Done Done Sent false
32823 9692 Acked Read 4 Done Done Sent false
32824 9693 Acked Write 6 Done Done Sent false
32825 9694 Acked Write 2 Done Done Sent false
32826 9695 Acked Read 10 Done Done Sent false
32827 9696 Acked Flush 0 Done Done Sent false
32828 9697 Acked Flush 0 Done Done Sent false
32829 9698 Acked Read 2 Done Done Sent false
32830 9699 Acked Write 8 Done Done Sent false
32831 9700 Acked Read 10 Done Done Sent false
32832 9701 Acked Read 6 Done Done Sent false
32833 9702 Acked Read 10 Done Done Sent false
32834 9703 Acked Write 9 Done Done Sent false
32835 9704 Acked Write 9 Done Done Sent false
32836 9705 Acked Flush 0 Done Done Sent false
32837 9706 Acked Write 6 Done Done Sent false
32838 9707 Acked Read 3 Done Done Sent false
32839 9708 Acked Write 6 Done Done Sent false
32840 9709 Acked Read 9 Done Done Sent false
32841 9710 Acked Write 3 Done Done Sent false
32842 9711 Acked Write 1 Done Done Sent false
32843 9712 Acked Write 9 Done Done Sent false
32844 9713 Acked Flush 0 Done Done Sent false
32845 9714 Acked Read 2 Done Done Sent false
32846 9715 Acked Write 4 Done Done Sent false
32847 9716 Acked Write 6 Done Done Sent false
32848 9717 Acked Write 1 Done Done Sent false
32849 9718 Acked Write 10 Done Done Sent false
32850 9719 Acked Read 5 Done Done Sent false
32851 9720 Acked Write 7 Done Done Sent false
32852 9721 Acked Write 9 Done Done Sent false
32853 9722 Acked Read 6 Done Done Sent false
32854 9723 Acked Read 6 Done Done Sent false
32855 9724 Acked Write 4 Done Done Sent false
32856 9725 Acked Write 1 Done Done Sent false
32857 9726 Acked Write 7 Done Done Sent false
32858 9727 Acked Write 7 Done Done Sent false
32859 9728 Acked Write 7 Done Done Sent false
32860 9729 Acked Read 1 Done Done Sent false
32861 9730 Acked Flush 0 Done Done Sent false
32862 9731 Acked Write 4 Done Done Sent false
32863 9732 Acked Read 10 Done Done Sent false
32864 9733 Acked Write 1 Done Done Sent false
32865 9734 Acked Write 7 Done Done Sent false
32866 9735 Acked Read 3 Done Done Sent false
32867 9736 Acked Write 3 Done Done Sent false
32868 9737 Acked Read 10 Done Done Sent false
32869 9738 Acked Read 4 Done Done Sent false
32870 9739 Acked Read 7 Done Done Sent false
32871 9740 Acked Write 2 Done Done Sent false
32872 9741 Acked Flush 0 Done Done Sent false
32873 9742 Acked Write 6 Done Done Sent false
32874 9743 Acked Flush 0 Done Done Sent false
32875 9744 Acked Read 7 Done Done Sent false
32876 9745 Acked Read 4 Done Done Sent false
32877 9746 Acked Read 1 Done Done Sent false
32878 9747 Acked Read 10 Done Done Sent false
32879 9748 Acked Write 10 Done Done Sent false
32880 9749 Acked Read 10 Done Done Sent false
32881 9750 Acked Read 4 Done Done Sent false
32882 9751 Acked Read 3 Done Done Sent false
32883 9752 Acked Write 1 Done Done Sent false
32884 9753 Acked Write 2 Done Done Sent false
32885 9754 Acked Write 8 Done Done Sent false
32886 9755 Acked Read 3 Done Done Sent false
32887 9756 Acked Read 7 Done Done Sent false
32888 9757 Acked Read 2 Done Done Sent false
32889 9758 Acked Write 8 Done Done Sent false
32890 9759 Acked Read 6 Done Done Sent false
32891 9760 Acked Write 8 Done Done Sent false
32892 9761 Acked Read 5 Done Done Sent false
32893 9762 Acked Read 9 Done Done Sent false
32894 9763 Acked Write 10 Done Done Sent false
32895 9764 Acked Write 6 Done Done Sent false
32896 9765 Acked Read 7 Done Done Sent false
32897 9766 Acked Write 4 Done Done Sent false
32898 9767 Acked Flush 0 Done Done Sent false
32899 9768 Acked Flush 0 Done Done Sent false
32900 STATES DS:0 DS:1 DS:2 TOTAL
32901 Sent 0 0 301 301
32902 Done 301 301 0 602
32903 Skipped 0 0 0 0
32904 Error 0 0 0 0
32905 Last Flush: 9768 9768 9467
32906 Downstairs last five completed: 9467 9466 9465 9464 9463
32907 Upstairs last five completed: 9768 9767 9766 9765 9764
32908 CLIENT: Up:0 ds:301 act:2
329092024-11-07T14:10:12.106ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
329102024-11-07T14:10:12.106ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
329112024-11-07T14:10:12.106ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
329122024-11-07T14:10:12.106ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
329132024-11-07T14:10:12.106ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
32914 [8830] got start action
32915 [8830] I am going to restart
32916 Starting downstairs at port 8830
32917 Make output file at "/tmp/dsc/downstairs-8830.txt"
32918 [8830][2] reports Starting
32919 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1228)
32920 [8830][2] reports Running
32921 ----------------------------------------------------------------
32922 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32923 States: Active Active Offline
32924 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32925 9468 Acked Read 7 Done Done Sent false
32926 9469 Acked Write 4 Done Done Sent false
32927 9470 Acked Write 3 Done Done Sent false
32928 9471 Acked Write 3 Done Done Sent false
32929 9472 Acked Write 9 Done Done Sent false
32930 9473 Acked Read 1 Done Done Sent false
32931 9474 Acked Write 5 Done Done Sent false
32932 9475 Acked Read 2 Done Done Sent false
32933 9476 Acked Read 6 Done Done Sent false
32934 9477 Acked Read 4 Done Done Sent false
32935 9478 Acked Read 8 Done Done Sent false
32936 9479 Acked Write 1 Done Done Sent false
32937 9480 Acked Read 1 Done Done Sent false
32938 9481 Acked Flush 0 Done Done Sent false
32939 9482 Acked Write 1 Done Done Sent false
32940 9483 Acked Read 8 Done Done Sent false
32941 9484 Acked Write 1 Done Done Sent false
32942 9485 Acked Write 7 Done Done Sent false
32943 9486 Acked Read 3 Done Done Sent false
32944 9487 Acked Read 4 Done Done Sent false
32945 9488 Acked Write 1 Done Done Sent false
32946 9489 Acked Read 8 Done Done Sent false
32947 9490 Acked Write 7 Done Done Sent false
32948 9491 Acked Read 5 Done Done Sent false
32949 9492 Acked Flush 0 Done Done Sent false
32950 9493 Acked Write 6 Done Done Sent false
32951 9494 Acked Flush 0 Done Done Sent false
32952 9495 Acked Read 6 Done Done Sent false
32953 9496 Acked Write 9 Done Done Sent false
32954 9497 Acked Read 8 Done Done Sent false
32955 9498 Acked Read 3 Done Done Sent false
32956 9499 Acked Read 9 Done Done Sent false
32957 9500 Acked Write 6 Done Done Sent false
32958 9501 Acked Write 8 Done Done Sent false
32959 9502 Acked Read 5 Done Done Sent false
32960 9503 Acked Read 7 Done Done Sent false
32961 9504 Acked Read 2 Done Done Sent false
32962 9505 Acked Write 1 Done Done Sent false
32963 9506 Acked Read 3 Done Done Sent false
32964 9507 Acked Write 4 Done Done Sent false
32965 9508 Acked Flush 0 Done Done Sent false
32966 9509 Acked Read 2 Done Done Sent false
32967 9510 Acked Write 4 Done Done Sent false
32968 9511 Acked Write 6 Done Done Sent false
32969 9512 Acked Write 6 Done Done Sent false
32970 9513 Acked Write 3 Done Done Sent false
32971 9514 Acked Read 2 Done Done Sent false
32972 9515 Acked Read 9 Done Done Sent false
32973 9516 Acked Read 3 Done Done Sent false
32974 9517 Acked Read 5 Done Done Sent false
32975 9518 Acked Read 4 Done Done Sent false
32976 9519 Acked Read 9 Done Done Sent false
32977 9520 Acked Write 4 Done Done Sent false
32978 9521 Acked Write 6 Done Done Sent false
32979 9522 Acked Read 7 Done Done Sent false
32980 9523 Acked Write 9 Done Done Sent false
32981 9524 Acked Write 4 Done Done Sent false
32982 9525 Acked Flush 0 Done Done Sent false
32983 9526 Acked Write 4 Done Done Sent false
32984 9527 Acked Write 2 Done Done Sent false
32985 9528 Acked Write 2 Done Done Sent false
32986 9529 Acked Write 5 Done Done Sent false
32987 9530 Acked Write 7 Done Done Sent false
32988 9531 Acked Write 3 Done Done Sent false
32989 9532 Acked Read 7 Done Done Sent false
32990 9533 Acked Write 9 Done Done Sent false
32991 9534 Acked Write 5 Done Done Sent false
32992 9535 Acked Read 2 Done Done Sent false
32993 9536 Acked Write 4 Done Done Sent false
32994 9537 Acked Read 2 Done Done Sent false
32995 9538 Acked Write 9 Done Done Sent false
32996 9539 Acked Read 5 Done Done Sent false
32997 9540 Acked Read 5 Done Done Sent false
32998 9541 Acked Write 1 Done Done Sent false
32999 9542 Acked Read 9 Done Done Sent false
33000 9543 Acked Write 6 Done Done Sent false
33001 9544 Acked Read 8 Done Done Sent false
33002 9545 Acked Flush 0 Done Done Sent false
33003 9546 Acked Write 8 Done Done Sent false
33004 9547 Acked Read 7 Done Done Sent false
33005 9548 Acked Flush 0 Done Done Sent false
33006 9549 Acked Write 8 Done Done Sent false
33007 9550 Acked Read 10 Done Done Sent false
33008 9551 Acked Read 5 Done Done Sent false
33009 9552 Acked Read 4 Done Done Sent false
33010 9553 Acked Write 6 Done Done Sent false
33011 9554 Acked Write 1 Done Done Sent false
33012 9555 Acked Write 1 Done Done Sent false
33013 9556 Acked Read 8 Done Done Sent false
33014 9557 Acked Read 1 Done Done Sent false
33015 9558 Acked Write 9 Done Done Sent false
33016 9559 Acked Read 2 Done Done Sent false
33017 9560 Acked Read 6 Done Done Sent false
33018 9561 Acked Flush 0 Done Done Sent false
33019 9562 Acked Flush 0 Done Done Sent false
33020 9563 Acked Write 5 Done Done Sent false
33021 9564 Acked Write 7 Done Done Sent false
33022 9565 Acked Read 5 Done Done Sent false
33023 9566 Acked Read 8 Done Done Sent false
33024 9567 Acked Flush 0 Done Done Sent false
33025 9568 Acked Write 2 Done Done Sent false
33026 9569 Acked Read 9 Done Done Sent false
33027 9570 Acked Flush 0 Done Done Sent false
33028 9571 Acked Read 7 Done Done Sent false
33029 9572 Acked Write 6 Done Done Sent false
33030 9573 Acked Write 7 Done Done Sent false
33031 9574 Acked Flush 0 Done Done Sent false
33032 9575 Acked Read 8 Done Done Sent false
33033 9576 Acked Read 6 Done Done Sent false
33034 9577 Acked Read 1 Done Done Sent false
33035 9578 Acked Write 10 Done Done Sent false
33036 9579 Acked Read 2 Done Done Sent false
33037 9580 Acked Read 5 Done Done Sent false
33038 9581 Acked Read 1 Done Done Sent false
33039 9582 Acked Write 10 Done Done Sent false
33040 9583 Acked Write 8 Done Done Sent false
33041 9584 Acked Read 4 Done Done Sent false
33042 9585 Acked Read 2 Done Done Sent false
33043 9586 Acked Read 6 Done Done Sent false
33044 9587 Acked Write 8 Done Done Sent false
33045 9588 Acked Write 10 Done Done Sent false
33046 9589 Acked Write 7 Done Done Sent false
33047 9590 Acked Read 6 Done Done Sent false
33048 9591 Acked Write 7 Done Done Sent false
33049 9592 Acked Flush 0 Done Done Sent false
33050 9593 Acked Write 5 Done Done Sent false
33051 9594 Acked Write 5 Done Done Sent false
33052 9595 Acked Read 10 Done Done Sent false
33053 9596 Acked Read 3 Done Done Sent false
33054 9597 Acked Write 5 Done Done Sent false
33055 9598 Acked Flush 0 Done Done Sent false
33056 9599 Acked Write 8 Done Done Sent false
33057 9600 Acked Write 6 Done Done Sent false
33058 9601 Acked Write 3 Done Done Sent false
33059 9602 Acked Read 10 Done Done Sent false
33060 9603 Acked Write 5 Done Done Sent false
33061 9604 Acked Read 4 Done Done Sent false
33062 9605 Acked Read 8 Done Done Sent false
33063 9606 Acked Read 6 Done Done Sent false
33064 9607 Acked Write 10 Done Done Sent false
33065 9608 Acked Read 10 Done Done Sent false
33066 9609 Acked Read 6 Done Done Sent false
33067 9610 Acked Read 3 Done Done Sent false
33068 9611 Acked Read 6 Done Done Sent false
33069 9612 Acked Read 5 Done Done Sent false
33070 9613 Acked Read 1 Done Done Sent false
33071 9614 Acked Read 10 Done Done Sent false
33072 9615 Acked Read 9 Done Done Sent false
33073 9616 Acked Write 8 Done Done Sent false
33074 9617 Acked Write 9 Done Done Sent false
33075 9618 Acked Read 3 Done Done Sent false
33076 9619 Acked Write 4 Done Done Sent false
33077 9620 Acked Flush 0 Done Done Sent false
33078 9621 Acked Read 1 Done Done Sent false
33079 9622 Acked Flush 0 Done Done Sent false
33080 9623 Acked Write 10 Done Done Sent false
33081 9624 Acked Write 6 Done Done Sent false
33082 9625 Acked Read 1 Done Done Sent false
33083 9626 Acked Write 5 Done Done Sent false
33084 9627 Acked Flush 0 Done Done Sent false
33085 9628 Acked Read 10 Done Done Sent false
33086 9629 Acked Write 4 Done Done Sent false
33087 9630 Acked Flush 0 Done Done Sent false
33088 9631 Acked Write 9 Done Done Sent false
33089 9632 Acked Read 10 Done Done Sent false
33090 9633 Acked Write 9 Done Done Sent false
33091 9634 Acked Read 4 Done Done Sent false
33092 9635 Acked Write 5 Done Done Sent false
33093 9636 Acked Read 6 Done Done Sent false
33094 9637 Acked Write 9 Done Done Sent false
33095 9638 Acked Write 6 Done Done Sent false
33096 9639 Acked Read 9 Done Done Sent false
33097 9640 Acked Read 4 Done Done Sent false
33098 9641 Acked Write 8 Done Done Sent false
33099 9642 Acked Write 4 Done Done Sent false
33100 9643 Acked Read 9 Done Done Sent false
33101 9644 Acked Read 1 Done Done Sent false
33102 9645 Acked Flush 0 Done Done Sent false
33103 9646 Acked Read 4 Done Done Sent false
33104 9647 Acked Read 9 Done Done Sent false
33105 9648 Acked Read 4 Done Done Sent false
33106 9649 Acked Read 10 Done Done Sent false
33107 9650 Acked Write 10 Done Done Sent false
33108 9651 Acked Flush 0 Done Done Sent false
33109 9652 Acked Write 4 Done Done Sent false
33110 9653 Acked Read 8 Done Done Sent false
33111 9654 Acked Write 9 Done Done Sent false
33112 9655 Acked Read 9 Done Done Sent false
33113 9656 Acked Write 10 Done Done Sent false
33114 9657 Acked Write 2 Done Done Sent false
33115 9658 Acked Write 5 Done Done Sent false
33116 9659 Acked Read 3 Done Done Sent false
33117 9660 Acked Write 5 Done Done Sent false
33118 9661 Acked Read 8 Done Done Sent false
33119 9662 Acked Flush 0 Done Done Sent false
33120 9663 Acked Read 7 Done Done Sent false
33121 9664 Acked Read 10 Done Done Sent false
33122 9665 Acked Read 8 Done Done Sent false
33123 9666 Acked Write 10 Done Done Sent false
33124 9667 Acked Write 5 Done Done Sent false
33125 9668 Acked Write 8 Done Done Sent false
33126 9669 Acked Read 4 Done Done Sent false
33127 9670 Acked Read 1 Done Done Sent false
33128 9671 Acked Write 9 Done Done Sent false
33129 9672 Acked Read 9 Done Done Sent false
33130 9673 Acked Write 7 Done Done Sent false
33131 9674 Acked Flush 0 Done Done Sent false
33132 9675 Acked Write 3 Done Done Sent false
33133 9676 Acked Write 5 Done Done Sent false
33134 9677 Acked Read 5 Done Done Sent false
33135 9678 Acked Read 5 Done Done Sent false
33136 9679 Acked Read 2 Done Done Sent false
33137 9680 Acked Read 10 Done Done Sent false
33138 9681 Acked Read 1 Done Done Sent false
33139 9682 Acked Flush 0 Done Done Sent false
33140 9683 Acked Read 10 Done Done Sent false
33141 9684 Acked Read 7 Done Done Sent false
33142 9685 Acked Read 3 Done Done Sent false
33143 9686 Acked Write 5 Done Done Sent false
33144 9687 Acked Write 3 Done Done Sent false
33145 9688 Acked Write 3 Done Done Sent false
33146 9689 Acked Read 2 Done Done Sent false
33147 9690 Acked Write 6 Done Done Sent false
33148 9691 Acked Write 6 Done Done Sent false
33149 9692 Acked Read 4 Done Done Sent false
33150 9693 Acked Write 6 Done Done Sent false
33151 9694 Acked Write 2 Done Done Sent false
33152 9695 Acked Read 10 Done Done Sent false
33153 9696 Acked Flush 0 Done Done Sent false
33154 9697 Acked Flush 0 Done Done Sent false
33155 9698 Acked Read 2 Done Done Sent false
33156 9699 Acked Write 8 Done Done Sent false
33157 9700 Acked Read 10 Done Done Sent false
33158 9701 Acked Read 6 Done Done Sent false
33159 9702 Acked Read 10 Done Done Sent false
33160 9703 Acked Write 9 Done Done Sent false
33161 9704 Acked Write 9 Done Done Sent false
33162 9705 Acked Flush 0 Done Done Sent false
33163 9706 Acked Write 6 Done Done Sent false
33164 9707 Acked Read 3 Done Done Sent false
33165 9708 Acked Write 6 Done Done Sent false
33166 9709 Acked Read 9 Done Done Sent false
33167 9710 Acked Write 3 Done Done Sent false
33168 9711 Acked Write 1 Done Done Sent false
33169 9712 Acked Write 9 Done Done Sent false
33170 9713 Acked Flush 0 Done Done Sent false
33171 9714 Acked Read 2 Done Done Sent false
33172 9715 Acked Write 4 Done Done Sent false
33173 9716 Acked Write 6 Done Done Sent false
33174 9717 Acked Write 1 Done Done Sent false
33175 9718 Acked Write 10 Done Done Sent false
33176 9719 Acked Read 5 Done Done Sent false
33177 9720 Acked Write 7 Done Done Sent false
33178 9721 Acked Write 9 Done Done Sent false
33179 9722 Acked Read 6 Done Done Sent false
33180 9723 Acked Read 6 Done Done Sent false
33181 9724 Acked Write 4 Done Done Sent false
33182 9725 Acked Write 1 Done Done Sent false
33183 9726 Acked Write 7 Done Done Sent false
33184 9727 Acked Write 7 Done Done Sent false
33185 9728 Acked Write 7 Done Done Sent false
33186 9729 Acked Read 1 Done Done Sent false
33187 9730 Acked Flush 0 Done Done Sent false
33188 9731 Acked Write 4 Done Done Sent false
33189 9732 Acked Read 10 Done Done Sent false
33190 9733 Acked Write 1 Done Done Sent false
33191 9734 Acked Write 7 Done Done Sent false
33192 9735 Acked Read 3 Done Done Sent false
33193 9736 Acked Write 3 Done Done Sent false
33194 9737 Acked Read 10 Done Done Sent false
33195 9738 Acked Read 4 Done Done Sent false
33196 9739 Acked Read 7 Done Done Sent false
33197 9740 Acked Write 2 Done Done Sent false
33198 9741 Acked Flush 0 Done Done Sent false
33199 9742 Acked Write 6 Done Done Sent false
33200 9743 Acked Flush 0 Done Done Sent false
33201 9744 Acked Read 7 Done Done Sent false
33202 9745 Acked Read 4 Done Done Sent false
33203 9746 Acked Read 1 Done Done Sent false
33204 9747 Acked Read 10 Done Done Sent false
33205 9748 Acked Write 10 Done Done Sent false
33206 9749 Acked Read 10 Done Done Sent false
33207 9750 Acked Read 4 Done Done Sent false
33208 9751 Acked Read 3 Done Done Sent false
33209 9752 Acked Write 1 Done Done Sent false
33210 9753 Acked Write 2 Done Done Sent false
33211 9754 Acked Write 8 Done Done Sent false
33212 9755 Acked Read 3 Done Done Sent false
33213 9756 Acked Read 7 Done Done Sent false
33214 9757 Acked Read 2 Done Done Sent false
33215 9758 Acked Write 8 Done Done Sent false
33216 9759 Acked Read 6 Done Done Sent false
33217 9760 Acked Write 8 Done Done Sent false
33218 9761 Acked Read 5 Done Done Sent false
33219 9762 Acked Read 9 Done Done Sent false
33220 9763 Acked Write 10 Done Done Sent false
33221 9764 Acked Write 6 Done Done Sent false
33222 9765 Acked Read 7 Done Done Sent false
33223 9766 Acked Write 4 Done Done Sent false
33224 9767 Acked Flush 0 Done Done Sent false
33225 9768 Acked Flush 0 Done Done Sent false
33226 STATES DS:0 DS:1 DS:2 TOTAL
33227 Sent 0 0 301 301
33228 Done 301 301 0 602
33229 Skipped 0 0 0 0
33230 Error 0 0 0 0
33231 Last Flush: 9768 9768 9467
33232 Downstairs last five completed: 9467 9466 9465 9464 9463
33233 Upstairs last five completed: 9768 9767 9766 9765 9764
33234 CLIENT: Up:0 ds:301 act:2
33235 ----------------------------------------------------------------
33236 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33237 States: Active Active Offline
33238 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33239 9468 Acked Read 7 Done Done Sent false
33240 9469 Acked Write 4 Done Done Sent false
33241 9470 Acked Write 3 Done Done Sent false
33242 9471 Acked Write 3 Done Done Sent false
33243 9472 Acked Write 9 Done Done Sent false
33244 9473 Acked Read 1 Done Done Sent false
33245 9474 Acked Write 5 Done Done Sent false
33246 9475 Acked Read 2 Done Done Sent false
33247 9476 Acked Read 6 Done Done Sent false
33248 9477 Acked Read 4 Done Done Sent false
33249 9478 Acked Read 8 Done Done Sent false
33250 9479 Acked Write 1 Done Done Sent false
33251 9480 Acked Read 1 Done Done Sent false
33252 9481 Acked Flush 0 Done Done Sent false
33253 9482 Acked Write 1 Done Done Sent false
33254 9483 Acked Read 8 Done Done Sent false
33255 9484 Acked Write 1 Done Done Sent false
33256 9485 Acked Write 7 Done Done Sent false
33257 9486 Acked Read 3 Done Done Sent false
33258 9487 Acked Read 4 Done Done Sent false
33259 9488 Acked Write 1 Done Done Sent false
33260 9489 Acked Read 8 Done Done Sent false
33261 9490 Acked Write 7 Done Done Sent false
33262 9491 Acked Read 5 Done Done Sent false
33263 9492 Acked Flush 0 Done Done Sent false
33264 9493 Acked Write 6 Done Done Sent false
33265 9494 Acked Flush 0 Done Done Sent false
33266 9495 Acked Read 6 Done Done Sent false
33267 9496 Acked Write 9 Done Done Sent false
33268 9497 Acked Read 8 Done Done Sent false
33269 9498 Acked Read 3 Done Done Sent false
33270 9499 Acked Read 9 Done Done Sent false
33271 9500 Acked Write 6 Done Done Sent false
33272 9501 Acked Write 8 Done Done Sent false
33273 9502 Acked Read 5 Done Done Sent false
33274 9503 Acked Read 7 Done Done Sent false
33275 9504 Acked Read 2 Done Done Sent false
33276 9505 Acked Write 1 Done Done Sent false
33277 9506 Acked Read 3 Done Done Sent false
33278 9507 Acked Write 4 Done Done Sent false
33279 9508 Acked Flush 0 Done Done Sent false
33280 9509 Acked Read 2 Done Done Sent false
33281 9510 Acked Write 4 Done Done Sent false
33282 9511 Acked Write 6 Done Done Sent false
33283 9512 Acked Write 6 Done Done Sent false
33284 9513 Acked Write 3 Done Done Sent false
33285 9514 Acked Read 2 Done Done Sent false
33286 9515 Acked Read 9 Done Done Sent false
33287 9516 Acked Read 3 Done Done Sent false
33288 9517 Acked Read 5 Done Done Sent false
33289 9518 Acked Read 4 Done Done Sent false
33290 9519 Acked Read 9 Done Done Sent false
33291 9520 Acked Write 4 Done Done Sent false
33292 9521 Acked Write 6 Done Done Sent false
33293 9522 Acked Read 7 Done Done Sent false
33294 9523 Acked Write 9 Done Done Sent false
33295 9524 Acked Write 4 Done Done Sent false
33296 9525 Acked Flush 0 Done Done Sent false
33297 9526 Acked Write 4 Done Done Sent false
33298 9527 Acked Write 2 Done Done Sent false
33299 9528 Acked Write 2 Done Done Sent false
33300 9529 Acked Write 5 Done Done Sent false
33301 9530 Acked Write 7 Done Done Sent false
33302 9531 Acked Write 3 Done Done Sent false
33303 9532 Acked Read 7 Done Done Sent false
33304 9533 Acked Write 9 Done Done Sent false
33305 9534 Acked Write 5 Done Done Sent false
33306 9535 Acked Read 2 Done Done Sent false
33307 9536 Acked Write 4 Done Done Sent false
33308 9537 Acked Read 2 Done Done Sent false
33309 9538 Acked Write 9 Done Done Sent false
33310 9539 Acked Read 5 Done Done Sent false
33311 9540 Acked Read 5 Done Done Sent false
33312 9541 Acked Write 1 Done Done Sent false
33313 9542 Acked Read 9 Done Done Sent false
33314 9543 Acked Write 6 Done Done Sent false
33315 9544 Acked Read 8 Done Done Sent false
33316 9545 Acked Flush 0 Done Done Sent false
33317 9546 Acked Write 8 Done Done Sent false
33318 9547 Acked Read 7 Done Done Sent false
33319 9548 Acked Flush 0 Done Done Sent false
33320 9549 Acked Write 8 Done Done Sent false
33321 9550 Acked Read 10 Done Done Sent false
33322 9551 Acked Read 5 Done Done Sent false
33323 9552 Acked Read 4 Done Done Sent false
33324 9553 Acked Write 6 Done Done Sent false
33325 9554 Acked Write 1 Done Done Sent false
33326 9555 Acked Write 1 Done Done Sent false
33327 9556 Acked Read 8 Done Done Sent false
33328 9557 Acked Read 1 Done Done Sent false
33329 9558 Acked Write 9 Done Done Sent false
33330 9559 Acked Read 2 Done Done Sent false
33331 9560 Acked Read 6 Done Done Sent false
33332 9561 Acked Flush 0 Done Done Sent false
33333 9562 Acked Flush 0 Done Done Sent false
33334 9563 Acked Write 5 Done Done Sent false
33335 9564 Acked Write 7 Done Done Sent false
33336 9565 Acked Read 5 Done Done Sent false
33337 9566 Acked Read 8 Done Done Sent false
33338 9567 Acked Flush 0 Done Done Sent false
33339 9568 Acked Write 2 Done Done Sent false
33340 9569 Acked Read 9 Done Done Sent false
33341 9570 Acked Flush 0 Done Done Sent false
33342 9571 Acked Read 7 Done Done Sent false
33343 9572 Acked Write 6 Done Done Sent false
33344 9573 Acked Write 7 Done Done Sent false
33345 9574 Acked Flush 0 Done Done Sent false
33346 9575 Acked Read 8 Done Done Sent false
33347 9576 Acked Read 6 Done Done Sent false
33348 9577 Acked Read 1 Done Done Sent false
33349 9578 Acked Write 10 Done Done Sent false
33350 9579 Acked Read 2 Done Done Sent false
33351 9580 Acked Read 5 Done Done Sent false
33352 9581 Acked Read 1 Done Done Sent false
33353 9582 Acked Write 10 Done Done Sent false
33354 9583 Acked Write 8 Done Done Sent false
33355 9584 Acked Read 4 Done Done Sent false
33356 9585 Acked Read 2 Done Done Sent false
33357 9586 Acked Read 6 Done Done Sent false
33358 9587 Acked Write 8 Done Done Sent false
33359 9588 Acked Write 10 Done Done Sent false
33360 9589 Acked Write 7 Done Done Sent false
33361 9590 Acked Read 6 Done Done Sent false
33362 9591 Acked Write 7 Done Done Sent false
33363 9592 Acked Flush 0 Done Done Sent false
33364 9593 Acked Write 5 Done Done Sent false
33365 9594 Acked Write 5 Done Done Sent false
33366 9595 Acked Read 10 Done Done Sent false
33367 9596 Acked Read 3 Done Done Sent false
33368 9597 Acked Write 5 Done Done Sent false
33369 9598 Acked Flush 0 Done Done Sent false
33370 9599 Acked Write 8 Done Done Sent false
33371 9600 Acked Write 6 Done Done Sent false
33372 9601 Acked Write 3 Done Done Sent false
33373 9602 Acked Read 10 Done Done Sent false
33374 9603 Acked Write 5 Done Done Sent false
33375 9604 Acked Read 4 Done Done Sent false
33376 9605 Acked Read 8 Done Done Sent false
33377 9606 Acked Read 6 Done Done Sent false
33378 9607 Acked Write 10 Done Done Sent false
33379 9608 Acked Read 10 Done Done Sent false
33380 9609 Acked Read 6 Done Done Sent false
33381 9610 Acked Read 3 Done Done Sent false
33382 9611 Acked Read 6 Done Done Sent false
33383 9612 Acked Read 5 Done Done Sent false
33384 9613 Acked Read 1 Done Done Sent false
33385 9614 Acked Read 10 Done Done Sent false
33386 9615 Acked Read 9 Done Done Sent false
33387 9616 Acked Write 8 Done Done Sent false
33388 9617 Acked Write 9 Done Done Sent false
33389 9618 Acked Read 3 Done Done Sent false
33390 9619 Acked Write 4 Done Done Sent false
33391 9620 Acked Flush 0 Done Done Sent false
33392 9621 Acked Read 1 Done Done Sent false
33393 9622 Acked Flush 0 Done Done Sent false
33394 9623 Acked Write 10 Done Done Sent false
33395 9624 Acked Write 6 Done Done Sent false
33396 9625 Acked Read 1 Done Done Sent false
33397 9626 Acked Write 5 Done Done Sent false
33398 9627 Acked Flush 0 Done Done Sent false
33399 9628 Acked Read 10 Done Done Sent false
33400 9629 Acked Write 4 Done Done Sent false
33401 9630 Acked Flush 0 Done Done Sent false
33402 9631 Acked Write 9 Done Done Sent false
33403 9632 Acked Read 10 Done Done Sent false
33404 9633 Acked Write 9 Done Done Sent false
33405 9634 Acked Read 4 Done Done Sent false
33406 9635 Acked Write 5 Done Done Sent false
33407 9636 Acked Read 6 Done Done Sent false
33408 9637 Acked Write 9 Done Done Sent false
33409 9638 Acked Write 6 Done Done Sent false
33410 9639 Acked Read 9 Done Done Sent false
33411 9640 Acked Read 4 Done Done Sent false
33412 9641 Acked Write 8 Done Done Sent false
33413 9642 Acked Write 4 Done Done Sent false
33414 9643 Acked Read 9 Done Done Sent false
33415 9644 Acked Read 1 Done Done Sent false
33416 9645 Acked Flush 0 Done Done Sent false
33417 9646 Acked Read 4 Done Done Sent false
33418 9647 Acked Read 9 Done Done Sent false
33419 9648 Acked Read 4 Done Done Sent false
33420 9649 Acked Read 10 Done Done Sent false
33421 9650 Acked Write 10 Done Done Sent false
33422 9651 Acked Flush 0 Done Done Sent false
33423 9652 Acked Write 4 Done Done Sent false
33424 9653 Acked Read 8 Done Done Sent false
33425 9654 Acked Write 9 Done Done Sent false
33426 9655 Acked Read 9 Done Done Sent false
33427 9656 Acked Write 10 Done Done Sent false
33428 9657 Acked Write 2 Done Done Sent false
33429 9658 Acked Write 5 Done Done Sent false
33430 9659 Acked Read 3 Done Done Sent false
33431 9660 Acked Write 5 Done Done Sent false
33432 9661 Acked Read 8 Done Done Sent false
33433 9662 Acked Flush 0 Done Done Sent false
33434 9663 Acked Read 7 Done Done Sent false
33435 9664 Acked Read 10 Done Done Sent false
33436 9665 Acked Read 8 Done Done Sent false
33437 9666 Acked Write 10 Done Done Sent false
33438 9667 Acked Write 5 Done Done Sent false
33439 9668 Acked Write 8 Done Done Sent false
33440 9669 Acked Read 4 Done Done Sent false
33441 9670 Acked Read 1 Done Done Sent false
33442 9671 Acked Write 9 Done Done Sent false
33443 9672 Acked Read 9 Done Done Sent false
33444 9673 Acked Write 7 Done Done Sent false
33445 9674 Acked Flush 0 Done Done Sent false
33446 9675 Acked Write 3 Done Done Sent false
33447 9676 Acked Write 5 Done Done Sent false
33448 9677 Acked Read 5 Done Done Sent false
33449 9678 Acked Read 5 Done Done Sent false
33450 9679 Acked Read 2 Done Done Sent false
33451 9680 Acked Read 10 Done Done Sent false
33452 9681 Acked Read 1 Done Done Sent false
33453 9682 Acked Flush 0 Done Done Sent false
33454 9683 Acked Read 10 Done Done Sent false
33455 9684 Acked Read 7 Done Done Sent false
33456 9685 Acked Read 3 Done Done Sent false
33457 9686 Acked Write 5 Done Done Sent false
33458 9687 Acked Write 3 Done Done Sent false
33459 9688 Acked Write 3 Done Done Sent false
33460 9689 Acked Read 2 Done Done Sent false
33461 9690 Acked Write 6 Done Done Sent false
33462 9691 Acked Write 6 Done Done Sent false
33463 9692 Acked Read 4 Done Done Sent false
33464 9693 Acked Write 6 Done Done Sent false
33465 9694 Acked Write 2 Done Done Sent false
33466 9695 Acked Read 10 Done Done Sent false
33467 9696 Acked Flush 0 Done Done Sent false
33468 9697 Acked Flush 0 Done Done Sent false
33469 9698 Acked Read 2 Done Done Sent false
33470 9699 Acked Write 8 Done Done Sent false
33471 9700 Acked Read 10 Done Done Sent false
33472 9701 Acked Read 6 Done Done Sent false
33473 9702 Acked Read 10 Done Done Sent false
33474 9703 Acked Write 9 Done Done Sent false
33475 9704 Acked Write 9 Done Done Sent false
33476 9705 Acked Flush 0 Done Done Sent false
33477 9706 Acked Write 6 Done Done Sent false
33478 9707 Acked Read 3 Done Done Sent false
33479 9708 Acked Write 6 Done Done Sent false
33480 9709 Acked Read 9 Done Done Sent false
33481 9710 Acked Write 3 Done Done Sent false
33482 9711 Acked Write 1 Done Done Sent false
33483 9712 Acked Write 9 Done Done Sent false
33484 9713 Acked Flush 0 Done Done Sent false
33485 9714 Acked Read 2 Done Done Sent false
33486 9715 Acked Write 4 Done Done Sent false
33487 9716 Acked Write 6 Done Done Sent false
33488 9717 Acked Write 1 Done Done Sent false
33489 9718 Acked Write 10 Done Done Sent false
33490 9719 Acked Read 5 Done Done Sent false
33491 9720 Acked Write 7 Done Done Sent false
33492 9721 Acked Write 9 Done Done Sent false
33493 9722 Acked Read 6 Done Done Sent false
33494 9723 Acked Read 6 Done Done Sent false
33495 9724 Acked Write 4 Done Done Sent false
33496 9725 Acked Write 1 Done Done Sent false
33497 9726 Acked Write 7 Done Done Sent false
33498 9727 Acked Write 7 Done Done Sent false
33499 9728 Acked Write 7 Done Done Sent false
33500 9729 Acked Read 1 Done Done Sent false
33501 9730 Acked Flush 0 Done Done Sent false
33502 9731 Acked Write 4 Done Done Sent false
33503 9732 Acked Read 10 Done Done Sent false
33504 9733 Acked Write 1 Done Done Sent false
33505 9734 Acked Write 7 Done Done Sent false
33506 9735 Acked Read 3 Done Done Sent false
33507 9736 Acked Write 3 Done Done Sent false
33508 9737 Acked Read 10 Done Done Sent false
33509 9738 Acked Read 4 Done Done Sent false
33510 9739 Acked Read 7 Done Done Sent false
33511 9740 Acked Write 2 Done Done Sent false
33512 9741 Acked Flush 0 Done Done Sent false
33513 9742 Acked Write 6 Done Done Sent false
33514 9743 Acked Flush 0 Done Done Sent false
33515 9744 Acked Read 7 Done Done Sent false
33516 9745 Acked Read 4 Done Done Sent false
33517 9746 Acked Read 1 Done Done Sent false
33518 9747 Acked Read 10 Done Done Sent false
33519 9748 Acked Write 10 Done Done Sent false
33520 9749 Acked Read 10 Done Done Sent false
33521 9750 Acked Read 4 Done Done Sent false
33522 9751 Acked Read 3 Done Done Sent false
33523 9752 Acked Write 1 Done Done Sent false
33524 9753 Acked Write 2 Done Done Sent false
33525 9754 Acked Write 8 Done Done Sent false
33526 9755 Acked Read 3 Done Done Sent false
33527 9756 Acked Read 7 Done Done Sent false
33528 9757 Acked Read 2 Done Done Sent false
33529 9758 Acked Write 8 Done Done Sent false
33530 9759 Acked Read 6 Done Done Sent false
33531 9760 Acked Write 8 Done Done Sent false
33532 9761 Acked Read 5 Done Done Sent false
33533 9762 Acked Read 9 Done Done Sent false
33534 9763 Acked Write 10 Done Done Sent false
33535 9764 Acked Write 6 Done Done Sent false
33536 9765 Acked Read 7 Done Done Sent false
33537 9766 Acked Write 4 Done Done Sent false
33538 9767 Acked Flush 0 Done Done Sent false
33539 9768 Acked Flush 0 Done Done Sent false
33540 STATES DS:0 DS:1 DS:2 TOTAL
33541 Sent 0 0 301 301
33542 Done 301 301 0 602
33543 Skipped 0 0 0 0
33544 Error 0 0 0 0
33545 Last Flush: 9768 9768 9467
33546 Downstairs last five completed: 9467 9466 9465 9464 9463
33547 Upstairs last five completed: 9768 9767 9766 9765 9764
33548 CLIENT: Up:0 ds:301 act:2
335492024-11-07T14:10:22.107ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335502024-11-07T14:10:22.107ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35114) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335512024-11-07T14:10:22.108ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335522024-11-07T14:10:22.109ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335532024-11-07T14:10:22.109ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335542024-11-07T14:10:22.109ZINFOcrucible: send last flush ID to this DS: 9467 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335552024-11-07T14:10:22.109ZINFOcrucible: Replied this last flush ID: 9467 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335562024-11-07T14:10:22.109ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335572024-11-07T14:10:22.109ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335582024-11-07T14:10:22.109ZINFOcrucible: [2] client re-new 301 jobs since flush 9467 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335592024-11-07T14:10:22.109ZINFOcrucible: [2] Replayed 301 jobs since flush: 9467 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
33560 ----------------------------------------------------------------
33561 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33562 Downstairs last five completed: 9768 9767 9766 9765 9764
33563 Upstairs last five completed: 9768 9767 9766 9765 9764
33564 CLIENT: Up:0 ds:0 act:3
33565 Replay: All jobs finished
33566 Main task has work to do, go find it
33567 got dsc Stop(1)
33568 stop 1
33569 Nov 07 14:10:22.415 [8820] Got stop action so:false kr:false
33570 INFO request completed, latency_us: 382, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 3602244d-fd80-43ae-80c3-d0be80500cec, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
335712024-11-07T14:10:22.416ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335722024-11-07T14:10:22.416ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335732024-11-07T14:10:22.416ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
335742024-11-07T14:10:22.416ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
33575 Nov 07 14:10:22.416 INFO request completed, latency_us: 154{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:10:22.41633876Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"client":"1","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
33576 , response_code: 200, uri: /state/cid/1, method: GET, req_id: d605e201-5849-4daa-ab5a-d7ccb7c27516, remote_addr: 127{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-07T14:10:22.416365428Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1199,"":"io task","client":"1","":"downstairs","session_id":"ac74b8f4-f0f2-4273-8fc2-10240de40f73"}
33577 .0.0.1:34100, local_addr: 127.0.0.1:9998
33578 [8820] Exited with: ExitStatus(unix_wait_status(9))
33579 [8820][1] reports Exit
33580 Nov 07 14:10:26.418 INFO request completed, latency_us: 224, response_code: 200, uri: /state/cid/1, method: GET, req_id: bb0edd5d-13e2-4a4f-9621-8d7220f1c1e5, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
33581 001/300 Write block 2763 len 12288 data: 5 4 3
33582 002/300 Read block 1415 len 4096
33583 003/300 Flush
33584 004/300 Flush
33585 005/300 Read block 3296 len 32768
33586 006/300 Write block 2790 len 20480 data: 9 9 8 8 9
33587 007/300 Read block 1533 len 28672
33588 008/300 Write block 4896 len 8192 data: 9 8
33589 009/300 Flush
33590 010/300 Write block 4304 len 32768 data: 5 5 7 8 7 10 9 10
33591 011/300 Write block 1052 len 12288 data: 10 9 8
33592 012/300 Read block 1263 len 36864
33593 013/300 Write block 2572 len 32768 data: 4 3 3 5 6 6 7 7
33594 014/300 Write block 3453 len 16384 data: 3 2 3 4
33595 015/300 Write block 4605 len 32768 data: 6 5 4 5 3 4 4 3
33596 016/300 Flush
33597 017/300 Read block 335 len 16384
33598 018/300 Write block 838 len 32768 data: 4 3 3 3 2 2 2 3
33599 019/300 Write block 3002 len 12288 data: 2 3 2
33600 020/300 Read block 444 len 32768
33601 021/300 Write block 3299 len 36864 data: 4 4 4 4 4 2 6 6 7
33602 022/300 Write block 1162 len 36864 data: 5 5 6 6 4 6 7 7 7
33603 023/300 Read block 4527 len 20480
33604 024/300 Read block 750 len 40960
33605 025/300 Read block 4543 len 16384
33606 026/300 Read block 1928 len 8192
33607 027/300 Read block 1609 len 32768
33608 028/300 Write block 2135 len 8192 data: 6 4
33609 029/300 Write block 2812 len 12288 data: 5 6 6
33610 030/300 Write block 2166 len 12288 data: 8 8 8
33611 031/300 Write block 98 len 16384 data: 3 4 3 2
33612 032/300 Read block 3117 len 12288
33613 033/300 Write block 2187 len 12288 data: 6 5 5
33614 034/300 Read block 3773 len 4096
33615 035/300 Flush
33616 036/300 Read block 4406 len 28672
33617 037/300 Write block 4769 len 4096 data: 6
33618 038/300 Read block 3166 len 16384
33619 039/300 Read block 3104 len 4096
33620 040/300 Read block 3801 len 20480
33621 041/300 Read block 4358 len 24576
33622 042/300 Flush
33623 043/300 Write block 3944 len 20480 data: 3 3 3 4 3
33624 044/300 Read block 3544 len 12288
33625 045/300 Read block 1061 len 16384
33626 046/300 Read block 4355 len 20480
33627 047/300 Read block 4631 len 24576
33628 048/300 Read block 1534 len 28672
33629 049/300 Read block 720 len 8192
33630 050/300 Read block 4670 len 16384
33631 051/300 Write block 955 len 4096 data: 6
33632 052/300 Read block 2742 len 12288
33633 053/300 Read block 3818 len 40960
33634 054/300 Read block 661 len 12288
33635 055/300 Flush
33636 056/300 Read block 3371 len 4096
33637 057/300 Write block 2980 len 24576 data: 5 4 4 5 6 6
33638 058/300 Read block 4962 len 28672
33639 059/300 Read block 3502 len 40960
33640 060/300 Write block 117 len 20480 data: 5 5 6 6 6
33641 061/300 Read block 1297 len 8192
33642 062/300 Write block 2786 len 28672 data: 9 9 8 8 10 10 9
33643 063/300 Read block 4261 len 8192
33644 064/300 Flush
33645 065/300 Write block 3678 len 8192 data: 6 5
33646 066/300 Write block 3947 len 8192 data: 5 4
33647 067/300 Read block 4036 len 40960
33648 068/300 Write block 2704 len 28672 data: 5 3 4 4 5 6 7
33649 069/300 Write block 1900 len 36864 data: 6 7 7 8 7 5 5 4 5
33650 070/300 Read block 3042 len 8192
33651 071/300 Read block 3961 len 16384
33652 072/300 Read block 2880 len 36864
33653 073/300 Read block 1950 len 24576
33654 074/300 Read block 3726 len 32768
33655 075/300 Flush
33656 076/300 Read block 1539 len 24576
33657 077/300 Write block 3944 len 36864 data: 4 4 4 6 5 3 4 4 5
33658 078/300 Write block 4317 len 12288 data: 7 6 6
33659 079/300 Write block 3459 len 16384 data: 4 4 5 7
33660 080/300 Read block 4501 len 16384
33661 081/300 Write block 4478 len 20480 data: 6 7 7 9 10
33662 082/300 Write block 4969 len 16384 data: 6 6 5 4
33663 083/300 Write block 4877 len 36864 data: 6 5 4 5 5 6 5 5 5
33664 084/300 Flush
33665 085/300 Write block 570 len 12288 data: 7 6 4
33666 086/300 Write block 456 len 28672 data: 11 11 8 7 5 4 5
33667 087/300 Read block 2856 len 20480
33668 088/300 Read block 3609 len 8192
33669 089/300 Read block 3257 len 12288
33670 090/300 Write block 2150 len 4096 data: 8
33671 091/300 Write block 3213 len 20480 data: 5 5 3 3 2
33672 092/300 Write block 4958 len 16384 data: 10 10 10 7
33673 093/300 Write block 4017 len 24576 data: 6 6 4 5 7 7
33674 094/300 Write block 3886 len 24576 data: 6 6 6 7 8 7
33675 095/300 Read block 4798 len 40960
33676 096/300 Read block 1458 len 40960
33677 097/300 Write block 2755 len 16384 data: 7 7 7 5
33678 098/300 Write block 3610 len 16384 data: 4 4 3 2
33679 099/300 Write block 1972 len 16384 data: 7 7 7 5
33680 100/300 Read block 2087 len 40960
33681 101/300 Read block 996 len 32768
33682 102/300 Read block 2462 len 28672
33683 103/300 Write block 4076 len 16384 data: 7 7 7 5
33684 104/300 Write block 4543 len 16384 data: 7 7 6 5
33685 105/300 Read block 2175 len 20480
33686 106/300 Read block 618 len 32768
33687 107/300 Read block 2339 len 28672
33688 108/300 Read block 138 len 8192
33689 109/300 Read block 4659 len 12288
33690 110/300 Write block 3543 len 36864 data: 8 8 12 11 9 11 9 8 9
33691 111/300 Read block 4786 len 4096
33692 112/300 Read block 3674 len 12288
33693 113/300 Read block 1415 len 36864
33694 114/300 Write block 4511 len 40960 data: 6 7 7 7 3 3 3 3 4 5
33695 115/300 Read block 3403 len 32768
33696 116/300 Flush
33697 117/300 Write block 2111 len 24576 data: 4 6 6 6 6 4
33698 118/300 Write block 2909 len 8192 data: 7 7
33699 119/300 Write block 134 len 40960 data: 3 3 4 4 4 4 5 5 6 5
33700 120/300 Read block 1707 len 32768
33701 121/300 Write block 136 len 32768 data: 5 5 5 5 6 6 7 6
33702 122/300 Flush
33703 123/300 Flush
33704 124/300 Read block 3596 len 4096
33705 125/300 Read block 2342 len 4096
33706 126/300 Write block 1680 len 12288 data: 7 8 7
33707 127/300 Read block 298 len 4096
33708 128/300 Write block 91 len 4096 data: 5
33709 129/300 Write block 3569 len 28672 data: 8 6 5 4 4 4 4
33710 130/300 Read block 4557 len 28672
33711 131/300 Read block 1561 len 20480
33712 132/300 Read block 374 len 28672
33713 133/300 Read block 679 len 12288
33714 134/300 Write block 1842 len 16384 data: 6 8 7 6
33715 135/300 Read block 1939 len 24576
33716 136/300 Read block 1009 len 12288
33717 137/300 Write block 2531 len 16384 data: 5 6 5 6
33718 138/300 Read block 4379 len 4096
33719 139/300 Write block 773 len 8192 data: 7 6
33720 140/300 Write block 3059 len 8192 data: 5 4
33721 141/300 Read block 1256 len 4096
33722 142/300 Write block 2903 len 4096 data: 5
33723 143/300 Write block 4444 len 36864 data: 9 8 8 8 7 8 7 6 8
33724 144/300 Write block 4263 len 36864 data: 5 6 6 7 5 6 8 8 11
33725 145/300 Flush
33726 146/300 Read block 2493 len 16384
33727 147/300 Read block 2127 len 16384
33728 148/300 Write block 4652 len 24576 data: 7 5 4 6 5 4
33729 149/300 Read block 4647 len 8192
33730 150/300 Read block 497 len 4096
33731 151/300 Flush
33732 152/300 Write block 4273 len 12288 data: 10 7 7
33733 153/300 Read block 3970 len 36864
33734 154/300 Write block 642 len 12288 data: 5 5 6
33735 155/300 Write block 3156 len 32768 data: 6 5 4 4 4 4 3 3
33736 156/300 Write block 4719 len 20480 data: 4 4 4 4 6
33737 157/300 Write block 3259 len 36864 data: 5 5 5 7 5 6 6 6 8
33738 158/300 Read block 975 len 40960
33739 159/300 Read block 2498 len 24576
33740 160/300 Read block 891 len 40960
33741 161/300 Read block 1251 len 12288
33742 162/300 Write block 1604 len 36864 data: 9 8 6 6 5 4 4 6 5
33743 163/300 Read block 4928 len 36864
33744 164/300 Read block 4021 len 20480
33745 165/300 Read block 168 len 24576
33746 166/300 Write block 3965 len 24576 data: 8 7 8 8 7 6
33747 167/300 Read block 3445 len 4096
33748 168/300 Read block 2262 len 40960
33749 169/300 Read block 1756 len 20480
33750 170/300 Write block 3837 len 16384 data: 7 6 7 7
33751 171/300 Read block 2125 len 16384
33752 172/300 Read block 2943 len 8192
33753 173/300 Read block 918 len 24576
33754 174/300 Write block 3253 len 24576 data: 2 2 2 4 5 5
33755 175/300 Read block 2453 len 8192
33756 176/300 Read block 1393 len 32768
33757 177/300 Read block 237 len 20480
33758 178/300 Write block 1707 len 24576 data: 3 6 6 7 11 9
33759 179/300 Read block 3042 len 32768
33760 180/300 Flush
33761 181/300 Write block 1474 len 28672 data: 5 6 5 4 5 5 6
33762 182/300 Write block 1074 len 8192 data: 7 5
33763 183/300 Flush
33764 184/300 Write block 4391 len 4096 data: 7
33765 185/300 Read block 3673 len 8192
33766 186/300 Flush
33767 187/300 Flush
33768 188/300 Flush
33769 189/300 Write block 3353 len 12288 data: 6 6 5
33770 190/300 Read block 583 len 20480
33771 191/300 Read block 2587 len 24576
33772 192/300 Write block 242 len 16384 data: 5 5 5 4
33773 193/300 Write block 2872 len 12288 data: 3 3 4
33774 194/300 Read block 64 len 16384
33775 195/300 Write block 3644 len 28672 data: 5 3 5 4 4 4 5
33776 196/300 Write block 1919 len 20480 data: 11 9 7 6 5
33777 197/300 Write block 231 len 4096 data: 5
33778 198/300 Write block 2367 len 20480 data: 6 6 6 6 5
33779 199/300 Write block 1467 len 20480 data: 5 6 5 4 4
33780 200/300 Write block 185 len 36864 data: 8 10 9 9 9 9 7 7 6
33781 201/300 Read block 4548 len 20480
33782 202/300 Write block 2770 len 40960 data: 5 4 4 3 5 6 5 6 5 3
33783 203/300 Flush
33784 204/300 Write block 745 len 12288 data: 7 6 4
33785 205/300 Read block 558 len 36864
33786 206/300 Write block 4458 len 8192 data: 4 3
33787 207/300 Read block 3696 len 12288
33788 208/300 Write block 2008 len 24576 data: 10 8 8 8 8 10
33789 209/300 Write block 4273 len 24576 data: 11 8 8 6 7 7
33790 210/300 Write block 3380 len 12288 data: 8 9 8
33791 211/300 Write block 4230 len 20480 data: 9 7 7 6 7
33792 212/300 Read block 3194 len 20480
33793 213/300 Read block 2448 len 16384
33794 214/300 Flush
33795 215/300 Write block 1172 len 8192 data: 9 9
33796 216/300 Read block 1395 len 40960
33797 217/300 Write block 3354 len 36864 data: 7 6 5 4 4 3 3 4 4
33798 218/300 Read block 3509 len 12288
33799 219/300 Read block 1170 len 16384
33800 220/300 Write block 4660 len 8192 data: 6 8
33801 221/300 Flush
33802 222/300 Write block 3989 len 20480 data: 6 5 7 6 5
33803 223/300 Read block 4063 len 8192
33804 224/300 Read block 3496 len 8192
33805 225/300 Write block 2915 len 32768 data: 8 8 8 8 8 7 7 6
33806 226/300 Read block 2206 len 28672
33807 227/300 Read block 4596 len 20480
33808 228/300 Write block 1653 len 12288 data: 8 8 7
33809 229/300 Write block 2789 len 28672 data: 9 11 11 10 9 10 7
33810 230/300 Read block 3790 len 32768
33811 231/300 Write block 4946 len 20480 data: 5 5 6 6 7
33812 232/300 Write block 3534 len 12288 data: 9 7 7
33813 233/300 Read block 4408 len 40960
33814 234/300 Read block 1679 len 36864
33815 235/300 Read block 433 len 8192
33816 236/300 Read block 2853 len 8192
33817 237/300 Read block 4481 len 4096
33818 238/300 Read block 3340 len 24576
33819 239/300 Read block 2096 len 20480
33820 240/300 Read block 4022 len 36864
33821 241/300 Flush
33822 242/300 Flush
33823 243/300 Write block 1863 len 16384 data: 6 6 5 4
33824 244/300 Read block 3255 len 20480
33825 245/300 Read block 1653 len 20480
33826 246/300 Read block 250 len 20480
33827 247/300 Read block 1267 len 28672
33828 248/300 Read block 4540 len 12288
33829 249/300 Write block 4148 len 24576 data: 9 9 10 9 9 7
33830 250/300 Read block 4893 len 8192
33831 251/300 Flush
33832 252/300 Flush
33833 253/300 Read block 1643 len 36864
33834 254/300 Read block 4200 len 32768
33835 255/300 Flush
33836 256/300 Flush
33837 257/300 Write block 13 len 20480 data: 7 6 4 4 6
33838 258/300 Write block 3653 len 4096 data: 4
33839 259/300 Read block 4596 len 32768
33840 260/300 Flush
33841 261/300 Read block 1998 len 12288
33842 262/300 Read block 3685 len 32768
33843 263/300 Read block 901 len 8192
33844 264/300 Write block 2741 len 24576 data: 4 5 5 5 5 4
33845 265/300 Write block 786 len 36864 data: 4 4 4 4 6 5 6 6 8
33846 266/300 Write block 3336 len 32768 data: 8 7 8 9 8 8 9 6
33847 267/300 Write block 952 len 20480 data: 7 6 7 7 8
33848 268/300 Read block 4187 len 28672
33849 269/300 Read block 4253 len 28672
33850 270/300 Write block 4214 len 40960 data: 6 6 6 4 4 3 5 5 4 5
33851 271/300 Read block 3743 len 36864
33852 272/300 Read block 210 len 12288
33853 273/300 Read block 946 len 20480
33854 274/300 Write block 1766 len 16384 data: 6 7 7 9
33855 275/300 Write block 4743 len 4096 data: 5
33856 276/300 Write block 625 len 24576 data: 9 10 10 9 8 7
33857 277/300 Write block 1376 len 40960 data: 9 9 7 6 6 5 4 4 3 6
33858 278/300 Flush
33859 279/300 Read block 4492 len 12288
33860 280/300 Read block 4197 len 36864
33861 281/300 Write block 2695 len 12288 data: 6 6 6
33862 282/300 Read block 2982 len 12288
33863 283/300 Read block 1027 len 20480
33864 284/300 Read block 4594 len 16384
33865 285/300 Read block 4524 len 20480
33866 286/300 Write block 4031 len 12288 data: 3 3 4
33867 287/300 Write block 2230 len 16384 data: 8 7 5 5
33868 288/300 Write block 1652 len 20480 data: 8 9 9 8 8
33869 289/300 Read block 3560 len 32768
33870 290/300 Write block 3075 len 36864 data: 6 8 6 5 5 8 9 9 11
33871 291/300 Read block 4668 len 16384
33872 292/300 Write block 4885 len 4096 data: 6
33873 293/300 Read block 1190 len 20480
33874 294/300 Read block 4788 len 36864
33875 295/300 Flush
33876 296/300 Write block 1969 len 32768 data: 7 7 6 8 8 8 6 5
33877 297/300 Write block 681 len 28672 data: 6 4 5 5 5 5 3
33878 298/300 Read block 601 len 24576
33879 299/300 Write block 3036 len 36864 data: 4 3 4 5 3 4 4 3 3
33880 300/300 Write block 2699 len 8192 data: 7 7
33881 301/300 Write block 4146 len 32768 data: 6 6 10 10 11 10 10 8
33882 Main task has work to do, go find it
33883 got dsc Start(1)
33884 start 1
33885 Nov 07 14:10:26.692 INFO request completed, latency_us: 222, response_code: 204, uri: /start/cid/1, method: POST, req_id: f2596640-4315-4614-933b-198109ad01b5, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
33886 Replay: started 1, returned:Ok(())
33887 ----------------------------------------------------------------
33888 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33889 States: Active Offline Active
33890 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33891 9769 Acked Write 3 Done Sent Done false
33892 9770 Acked Read 1 Done Sent Done false
33893 9771 Acked Flush 0 Done Sent Done false
33894 9772 Acked Flush 0 Done Sent Done false
33895 9773 Acked Read 8 Done Sent Done false
33896 9774 Acked Write 5 Done Sent Done false
33897 9775 Acked Read 7 Done Sent Done false
33898 9776 Acked Write 2 Done Sent Done false
33899 9777 Acked Flush 0 Done Sent Done false
33900 9778 Acked Write 8 Done Sent Done false
33901 9779 Acked Write 3 Done Sent Done false
33902 9780 Acked Read 9 Done Sent Done false
33903 9781 Acked Write 8 Done Sent Done false
33904 9782 Acked Write 4 Done Sent Done false
33905 9783 Acked Write 8 Done Sent Done false
33906 9784 Acked Flush 0 Done Sent Done false
33907 9785 Acked Read 4 Done Sent Done false
33908 9786 Acked Write 8 Done Sent Done false
33909 9787 Acked Write 3 Done Sent Done false
33910 9788 Acked Read 8 Done Sent Done false
33911 9789 Acked Write 9 Done Sent Done false
33912 9790 Acked Write 9 Done Sent Done false
33913 9791 Acked Read 5 Done Sent Done false
33914 9792 Acked Read 10 Done Sent Done false
33915 9793 Acked Read 4 Done Sent Done false
33916 9794 Acked Read 2 Done Sent Done false
33917 9795 Acked Read 8 Done Sent Done false
33918 9796 Acked Write 2 Done Sent Done false
33919 9797 Acked Write 3 Done Sent Done false
33920 9798 Acked Write 3 Done Sent Done false
33921 9799 Acked Write 4 Done Sent Done false
33922 9800 Acked Read 3 Done Sent Done false
33923 9801 Acked Write 3 Done Sent Done false
33924 9802 Acked Read 1 Done Sent Done false
33925 9803 Acked Flush 0 Done Sent Done false
33926 9804 Acked Read 7 Done Sent Done false
33927 9805 Acked Write 1 Done Sent Done false
33928 9806 Acked Read 4 Done Sent Done false
33929 9807 Acked Read 1 Done Sent Done false
33930 9808 Acked Read 5 Done Sent Done false
33931 9809 Acked Read 6 Done Sent Done false
33932 9810 Acked Flush 0 Done Sent Done false
33933 9811 Acked Write 5 Done Sent Done false
33934 9812 Acked Read 3 Done Sent Done false
33935 9813 Acked Read 4 Done Sent Done false
33936 9814 Acked Read 5 Done Sent Done false
33937 9815 Acked Read 6 Done Sent Done false
33938 9816 Acked Read 7 Done Sent Done false
33939 9817 Acked Read 2 Done Sent Done false
33940 9818 Acked Read 4 Done Sent Done false
33941 9819 Acked Write 1 Done Sent Done false
33942 9820 Acked Read 3 Done Sent Done false
33943 9821 Acked Read 10 Done Sent Done false
33944 9822 Acked Read 3 Done Sent Done false
33945 9823 Acked Flush 0 Done Sent Done false
33946 9824 Acked Read 1 Done Sent Done false
33947 9825 Acked Write 6 Done Sent Done false
33948 9826 Acked Read 7 Done Sent Done false
33949 9827 Acked Read 10 Done Sent Done false
33950 9828 Acked Write 5 Done Sent Done false
33951 9829 Acked Read 2 Done Sent Done false
33952 9830 Acked Write 7 Done Sent Done false
33953 9831 Acked Read 2 Done Sent Done false
33954 9832 Acked Flush 0 Done Sent Done false
33955 9833 Acked Write 2 Done Sent Done false
33956 9834 Acked Write 2 Done Sent Done false
33957 9835 Acked Read 10 Done Sent Done false
33958 9836 Acked Write 7 Done Sent Done false
33959 9837 Acked Write 9 Done Sent Done false
33960 9838 Acked Read 2 Done Sent Done false
33961 9839 Acked Read 4 Done Sent Done false
33962 9840 Acked Read 9 Done Sent Done false
33963 9841 Acked Read 6 Done Sent Done false
33964 9842 Acked Read 8 Done Sent Done false
33965 9843 Acked Flush 0 Done Sent Done false
33966 9844 Acked Read 6 Done Sent Done false
33967 9845 Acked Write 9 Done Sent Done false
33968 9846 Acked Write 3 Done Sent Done false
33969 9847 Acked Write 4 Done Sent Done false
33970 9848 Acked Read 4 Done Sent Done false
33971 9849 Acked Write 5 Done Sent Done false
33972 9850 Acked Write 4 Done Sent Done false
33973 9851 Acked Write 9 Done Sent Done false
33974 9852 Acked Flush 0 Done Sent Done false
33975 9853 Acked Write 3 Done Sent Done false
33976 9854 Acked Write 7 Done Sent Done false
33977 9855 Acked Read 5 Done Sent Done false
33978 9856 Acked Read 2 Done Sent Done false
33979 9857 Acked Read 3 Done Sent Done false
33980 9858 Acked Write 1 Done Sent Done false
33981 9859 Acked Write 5 Done Sent Done false
33982 9860 Acked Write 4 Done Sent Done false
33983 9861 Acked Write 6 Done Sent Done false
33984 9862 Acked Write 6 Done Sent Done false
33985 9863 Acked Read 10 Done Sent Done false
33986 9864 Acked Read 10 Done Sent Done false
33987 9865 Acked Write 4 Done Sent Done false
33988 9866 Acked Write 4 Done Sent Done false
33989 9867 Acked Write 4 Done Sent Done false
33990 9868 Acked Read 10 Done Sent Done false
33991 9869 Acked Read 8 Done Sent Done false
33992 9870 Acked Read 7 Done Sent Done false
33993 9871 Acked Write 4 Done Sent Done false
33994 9872 Acked Write 4 Done Sent Done false
33995 9873 Acked Read 5 Done Sent Done false
33996 9874 Acked Read 8 Done Sent Done false
33997 9875 Acked Read 7 Done Sent Done false
33998 9876 Acked Read 2 Done Sent Done false
33999 9877 Acked Read 3 Done Sent Done false
34000 9878 Acked Write 9 Done Sent Done false
34001 9879 Acked Read 1 Done Sent Done false
34002 9880 Acked Read 3 Done Sent Done false
34003 9881 Acked Read 9 Done Sent Done false
34004 9882 Acked Write 10 Done Sent Done false
34005 9883 Acked Read 8 Done Sent Done false
34006 9884 Acked Flush 0 Done Sent Done false
34007 9885 Acked Write 6 Done Sent Done false
34008 9886 Acked Write 2 Done Sent Done false
34009 9887 Acked Write 10 Done Sent Done false
34010 9888 Acked Read 8 Done Sent Done false
34011 9889 Acked Write 8 Done Sent Done false
34012 9890 Acked Flush 0 Done Sent Done false
34013 9891 Acked Flush 0 Done Sent Done false
34014 9892 Acked Read 1 Done Sent Done false
34015 9893 Acked Read 1 Done Sent Done false
34016 9894 Acked Write 3 Done Sent Done false
34017 9895 Acked Read 1 Done Sent Done false
34018 9896 Acked Write 1 Done Sent Done false
34019 9897 Acked Write 7 Done Sent Done false
34020 9898 Acked Read 7 Done Sent Done false
34021 9899 Acked Read 5 Done Sent Done false
34022 9900 Acked Read 7 Done Sent Done false
34023 9901 Acked Read 3 Done Sent Done false
34024 9902 Acked Write 4 Done Sent Done false
34025 9903 Acked Read 6 Done Sent Done false
34026 9904 Acked Read 3 Done Sent Done false
34027 9905 Acked Write 4 Done Sent Done false
34028 9906 Acked Read 1 Done Sent Done false
34029 9907 Acked Write 2 Done Sent Done false
34030 9908 Acked Write 2 Done Sent Done false
34031 9909 Acked Read 1 Done Sent Done false
34032 9910 Acked Write 1 Done Sent Done false
34033 9911 Acked Write 9 Done Sent Done false
34034 9912 Acked Write 9 Done Sent Done false
34035 9913 Acked Flush 0 Done Sent Done false
34036 9914 Acked Read 4 Done Sent Done false
34037 9915 Acked Read 4 Done Sent Done false
34038 9916 Acked Write 6 Done Sent Done false
34039 9917 Acked Read 2 Done Sent Done false
34040 9918 Acked Read 1 Done Sent Done false
34041 9919 Acked Flush 0 Done Sent Done false
34042 9920 Acked Write 3 Done Sent Done false
34043 9921 Acked Read 9 Done Sent Done false
34044 9922 Acked Write 3 Done Sent Done false
34045 9923 Acked Write 8 Done Sent Done false
34046 9924 Acked Write 5 Done Sent Done false
34047 9925 Acked Write 9 Done Sent Done false
34048 9926 Acked Read 10 Done Sent Done false
34049 9927 Acked Read 6 Done Sent Done false
34050 9928 Acked Read 10 Done Sent Done false
34051 9929 Acked Read 3 Done Sent Done false
34052 9930 Acked Write 9 Done Sent Done false
34053 9931 Acked Read 9 Done Sent Done false
34054 9932 Acked Read 5 Done Sent Done false
34055 9933 Acked Read 6 Done Sent Done false
34056 9934 Acked Write 6 Done Sent Done false
34057 9935 Acked Read 1 Done Sent Done false
34058 9936 Acked Read 10 Done Sent Done false
34059 9937 Acked Read 5 Done Sent Done false
34060 9938 Acked Write 4 Done Sent Done false
34061 9939 Acked Read 4 Done Sent Done false
34062 9940 Acked Read 2 Done Sent Done false
34063 9941 Acked Read 6 Done Sent Done false
34064 9942 Acked Write 6 Done Sent Done false
34065 9943 Acked Read 2 Done Sent Done false
34066 9944 Acked Read 8 Done Sent Done false
34067 9945 Acked Read 5 Done Sent Done false
34068 9946 Acked Write 6 Done Sent Done false
34069 9947 Acked Read 8 Done Sent Done false
34070 9948 Acked Flush 0 Done Sent Done false
34071 9949 Acked Write 7 Done Sent Done false
34072 9950 Acked Write 2 Done Sent Done false
34073 9951 Acked Flush 0 Done Sent Done false
34074 9952 Acked Write 1 Done Sent Done false
34075 9953 Acked Read 2 Done Sent Done false
34076 9954 Acked Flush 0 Done Sent Done false
34077 9955 Acked Flush 0 Done Sent Done false
34078 9956 Acked Flush 0 Done Sent Done false
34079 9957 Acked Write 3 Done Sent Done false
34080 9958 Acked Read 5 Done Sent Done false
34081 9959 Acked Read 6 Done Sent Done false
34082 9960 Acked Write 4 Done Sent Done false
34083 9961 Acked Write 3 Done Sent Done false
34084 9962 Acked Read 4 Done Sent Done false
34085 9963 Acked Write 7 Done Sent Done false
34086 9964 Acked Write 5 Done Sent Done false
34087 9965 Acked Write 1 Done Sent Done false
34088 9966 Acked Write 5 Done Sent Done false
34089 9967 Acked Write 5 Done Sent Done false
34090 9968 Acked Write 9 Done Sent Done false
34091 9969 Acked Read 5 Done Sent Done false
34092 9970 Acked Write 10 Done Sent Done false
34093 9971 Acked Flush 0 Done Sent Done false
34094 9972 Acked Write 3 Done Sent Done false
34095 9973 Acked Read 9 Done Sent Done false
34096 9974 Acked Write 2 Done Sent Done false
34097 9975 Acked Read 3 Done Sent Done false
34098 9976 Acked Write 6 Done Sent Done false
34099 9977 Acked Write 6 Done Sent Done false
34100 9978 Acked Write 3 Done Sent Done false
34101 9979 Acked Write 5 Done Sent Done false
34102 9980 Acked Read 5 Done Sent Done false
34103 9981 Acked Read 4 Done Sent Done false
34104 9982 Acked Flush 0 Done Sent Done false
34105 9983 Acked Write 2 Done Sent Done false
34106 9984 Acked Read 10 Done Sent Done false
34107 9985 Acked Write 9 Done Sent Done false
34108 9986 Acked Read 3 Done Sent Done false
34109 9987 Acked Flush 0 Done Sent Done false
34110 9988 Acked Read 4 Done Sent Done false
34111 9989 Acked Write 2 Done Sent Done false
34112 9990 Acked Flush 0 Done Sent Done false
34113 9991 Acked Write 5 Done Sent Done false
34114 9992 Acked Read 2 Done Sent Done false
34115 9993 Acked Read 2 Done Sent Done false
34116 9994 Acked Write 8 Done Sent Done false
34117 9995 Acked Read 7 Done Sent Done false
34118 9996 Acked Read 5 Done Sent Done false
34119 9997 Acked Write 3 Done Sent Done false
34120 9998 Acked Write 7 Done Sent Done false
34121 9999 Acked Read 8 Done Sent Done false
34122 10000 Acked Write 5 Done Sent Done false
34123 10001 Acked Write 3 Done Sent Done false
34124 10002 Acked Read 10 Done Sent Done false
34125 10003 Acked Read 9 Done Sent Done false
34126 10004 Acked Read 2 Done Sent Done false
34127 10005 Acked Read 2 Done Sent Done false
34128 10006 Acked Read 1 Done Sent Done false
34129 10007 Acked Read 6 Done Sent Done false
34130 10008 Acked Read 5 Done Sent Done false
34131 10009 Acked Read 9 Done Sent Done false
34132 10010 Acked Flush 0 Done Sent Done false
34133 10011 Acked Flush 0 Done Sent Done false
34134 10012 Acked Write 4 Done Sent Done false
34135 10013 Acked Read 5 Done Sent Done false
34136 10014 Acked Read 5 Done Sent Done false
34137 10015 Acked Read 5 Done Sent Done false
34138 10016 Acked Read 7 Done Sent Done false
34139 10017 Acked Read 3 Done Sent Done false
34140 10018 Acked Write 6 Done Sent Done false
34141 10019 Acked Read 2 Done Sent Done false
34142 10020 Acked Flush 0 Done Sent Done false
34143 10021 Acked Flush 0 Done Sent Done false
34144 10022 Acked Read 9 Done Sent Done false
34145 10023 Acked Read 8 Done Sent Done false
34146 10024 Acked Flush 0 Done Sent Done false
34147 10025 Acked Flush 0 Done Sent Done false
34148 10026 Acked Write 5 Done Sent Done false
34149 10027 Acked Write 1 Done Sent Done false
34150 10028 Acked Read 8 Done Sent Done false
34151 10029 Acked Flush 0 Done Sent Done false
34152 10030 Acked Read 3 Done Sent Done false
34153 10031 Acked Read 8 Done Sent Done false
34154 10032 Acked Read 2 Done Sent Done false
34155 10033 Acked Write 6 Done Sent Done false
34156 10034 Acked Write 9 Done Sent Done false
34157 10035 Acked Write 8 Done Sent Done false
34158 10036 Acked Write 5 Done Sent Done false
34159 10037 Acked Read 7 Done Sent Done false
34160 10038 Acked Read 7 Done Sent Done false
34161 10039 Acked Write 10 Done Sent Done false
34162 10040 Acked Read 9 Done Sent Done false
34163 10041 Acked Read 3 Done Sent Done false
34164 10042 Acked Read 5 Done Sent Done false
34165 10043 Acked Write 4 Done Sent Done false
34166 10044 Acked Write 1 Done Sent Done false
34167 10045 Acked Write 6 Done Sent Done false
34168 10046 Acked Write 10 Done Sent Done false
34169 10047 Acked Flush 0 Done Sent Done false
34170 10048 Acked Read 3 Done Sent Done false
34171 10049 Acked Read 9 Done Sent Done false
34172 10050 Acked Write 3 Done Sent Done false
34173 10051 Acked Read 3 Done Sent Done false
34174 10052 Acked Read 5 Done Sent Done false
34175 10053 Acked Read 4 Done Sent Done false
34176 10054 Acked Read 5 Done Sent Done false
34177 10055 Acked Write 3 Done Sent Done false
34178 10056 Acked Write 4 Done Sent Done false
34179 10057 Acked Write 5 Done Sent Done false
34180 10058 Acked Read 8 Done Sent Done false
34181 10059 Acked Write 9 Done Sent Done false
34182 10060 Acked Read 4 Done Sent Done false
34183 10061 Acked Write 1 Done Sent Done false
34184 10062 Acked Read 5 Done Sent Done false
34185 10063 Acked Read 9 Done Sent Done false
34186 10064 Acked Flush 0 Done Sent Done false
34187 10065 Acked Write 8 Done Sent Done false
34188 10066 Acked Write 7 Done Sent Done false
34189 10067 Acked Read 6 Done Sent Done false
34190 10068 Acked Write 9 Done Sent Done false
34191 10069 Acked Write 2 Done Sent Done false
34192 10070 Acked Write 8 Done Sent Done false
34193 STATES DS:0 DS:1 DS:2 TOTAL
34194 Sent 0 302 0 302
34195 Done 302 0 302 604
34196 Skipped 0 0 0 0
34197 Error 0 0 0 0
34198 Last Flush: 10064 9768 10064
34199 Downstairs last five completed: 9768 9767 9766 9765 9764
34200 Upstairs last five completed: 10070 10069 10068 10067 10066
34201 CLIENT: Up:0 ds:302 act:2
34202 [8820] got start action
34203 [8820] I am going to restart
34204 Starting downstairs at port 8820
34205 Make output file at "/tmp/dsc/downstairs-8820.txt"
34206 [8820][1] reports Starting
34207 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1229)
34208 [8820][1] reports Running
34209 ----------------------------------------------------------------
34210 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
34211 States: Active Offline Active
34212 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34213 9769 Acked Write 3 Done Sent Done false
34214 9770 Acked Read 1 Done Sent Done false
34215 9771 Acked Flush 0 Done Sent Done false
34216 9772 Acked Flush 0 Done Sent Done false
34217 9773 Acked Read 8 Done Sent Done false
34218 9774 Acked Write 5 Done Sent Done false
34219 9775 Acked Read 7 Done Sent Done false
34220 9776 Acked Write 2 Done Sent Done false
34221 9777 Acked Flush 0 Done Sent Done false
34222 9778 Acked Write 8 Done Sent Done false
34223 9779 Acked Write 3 Done Sent Done false
34224 9780 Acked Read 9 Done Sent Done false
34225 9781 Acked Write 8 Done Sent Done false
34226 9782 Acked Write 4 Done Sent Done false
34227 9783 Acked Write 8 Done Sent Done false
34228 9784 Acked Flush 0 Done Sent Done false
34229 9785 Acked Read 4 Done Sent Done false
34230 9786 Acked Write 8 Done Sent Done false
34231 9787 Acked Write 3 Done Sent Done false
34232 9788 Acked Read 8 Done Sent Done false
34233 9789 Acked Write 9 Done Sent Done false
34234 9790 Acked Write 9 Done Sent Done false
34235 9791 Acked Read 5 Done Sent Done false
34236 9792 Acked Read 10 Done Sent Done false
34237 9793 Acked Read 4 Done Sent Done false
34238 9794 Acked Read 2 Done Sent Done false
34239 9795 Acked Read 8 Done Sent Done false
34240 9796 Acked Write 2 Done Sent Done false
34241 9797 Acked Write 3 Done Sent Done false
34242 9798 Acked Write 3 Done Sent Done false
34243 9799 Acked Write 4 Done Sent Done false
34244 9800 Acked Read 3 Done Sent Done false
34245 9801 Acked Write 3 Done Sent Done false
34246 9802 Acked Read 1 Done Sent Done false
34247 9803 Acked Flush 0 Done Sent Done false
34248 9804 Acked Read 7 Done Sent Done false
34249 9805 Acked Write 1 Done Sent Done false
34250 9806 Acked Read 4 Done Sent Done false
34251 9807 Acked Read 1 Done Sent Done false
34252 9808 Acked Read 5 Done Sent Done false
34253 9809 Acked Read 6 Done Sent Done false
34254 9810 Acked Flush 0 Done Sent Done false
34255 9811 Acked Write 5 Done Sent Done false
34256 9812 Acked Read 3 Done Sent Done false
34257 9813 Acked Read 4 Done Sent Done false
34258 9814 Acked Read 5 Done Sent Done false
34259 9815 Acked Read 6 Done Sent Done false
34260 9816 Acked Read 7 Done Sent Done false
34261 9817 Acked Read 2 Done Sent Done false
34262 9818 Acked Read 4 Done Sent Done false
34263 9819 Acked Write 1 Done Sent Done false
34264 9820 Acked Read 3 Done Sent Done false
34265 9821 Acked Read 10 Done Sent Done false
34266 9822 Acked Read 3 Done Sent Done false
34267 9823 Acked Flush 0 Done Sent Done false
34268 9824 Acked Read 1 Done Sent Done false
34269 9825 Acked Write 6 Done Sent Done false
34270 9826 Acked Read 7 Done Sent Done false
34271 9827 Acked Read 10 Done Sent Done false
34272 9828 Acked Write 5 Done Sent Done false
34273 9829 Acked Read 2 Done Sent Done false
34274 9830 Acked Write 7 Done Sent Done false
34275 9831 Acked Read 2 Done Sent Done false
34276 9832 Acked Flush 0 Done Sent Done false
34277 9833 Acked Write 2 Done Sent Done false
34278 9834 Acked Write 2 Done Sent Done false
34279 9835 Acked Read 10 Done Sent Done false
34280 9836 Acked Write 7 Done Sent Done false
34281 9837 Acked Write 9 Done Sent Done false
34282 9838 Acked Read 2 Done Sent Done false
34283 9839 Acked Read 4 Done Sent Done false
34284 9840 Acked Read 9 Done Sent Done false
34285 9841 Acked Read 6 Done Sent Done false
34286 9842 Acked Read 8 Done Sent Done false
34287 9843 Acked Flush 0 Done Sent Done false
34288 9844 Acked Read 6 Done Sent Done false
34289 9845 Acked Write 9 Done Sent Done false
34290 9846 Acked Write 3 Done Sent Done false
34291 9847 Acked Write 4 Done Sent Done false
34292 9848 Acked Read 4 Done Sent Done false
34293 9849 Acked Write 5 Done Sent Done false
34294 9850 Acked Write 4 Done Sent Done false
34295 9851 Acked Write 9 Done Sent Done false
34296 9852 Acked Flush 0 Done Sent Done false
34297 9853 Acked Write 3 Done Sent Done false
34298 9854 Acked Write 7 Done Sent Done false
34299 9855 Acked Read 5 Done Sent Done false
34300 9856 Acked Read 2 Done Sent Done false
34301 9857 Acked Read 3 Done Sent Done false
34302 9858 Acked Write 1 Done Sent Done false
34303 9859 Acked Write 5 Done Sent Done false
34304 9860 Acked Write 4 Done Sent Done false
34305 9861 Acked Write 6 Done Sent Done false
34306 9862 Acked Write 6 Done Sent Done false
34307 9863 Acked Read 10 Done Sent Done false
34308 9864 Acked Read 10 Done Sent Done false
34309 9865 Acked Write 4 Done Sent Done false
34310 9866 Acked Write 4 Done Sent Done false
34311 9867 Acked Write 4 Done Sent Done false
34312 9868 Acked Read 10 Done Sent Done false
34313 9869 Acked Read 8 Done Sent Done false
34314 9870 Acked Read 7 Done Sent Done false
34315 9871 Acked Write 4 Done Sent Done false
34316 9872 Acked Write 4 Done Sent Done false
34317 9873 Acked Read 5 Done Sent Done false
34318 9874 Acked Read 8 Done Sent Done false
34319 9875 Acked Read 7 Done Sent Done false
34320 9876 Acked Read 2 Done Sent Done false
34321 9877 Acked Read 3 Done Sent Done false
34322 9878 Acked Write 9 Done Sent Done false
34323 9879 Acked Read 1 Done Sent Done false
34324 9880 Acked Read 3 Done Sent Done false
34325 9881 Acked Read 9 Done Sent Done false
34326 9882 Acked Write 10 Done Sent Done false
34327 9883 Acked Read 8 Done Sent Done false
34328 9884 Acked Flush 0 Done Sent Done false
34329 9885 Acked Write 6 Done Sent Done false
34330 9886 Acked Write 2 Done Sent Done false
34331 9887 Acked Write 10 Done Sent Done false
34332 9888 Acked Read 8 Done Sent Done false
34333 9889 Acked Write 8 Done Sent Done false
34334 9890 Acked Flush 0 Done Sent Done false
34335 9891 Acked Flush 0 Done Sent Done false
34336 9892 Acked Read 1 Done Sent Done false
34337 9893 Acked Read 1 Done Sent Done false
34338 9894 Acked Write 3 Done Sent Done false
34339 9895 Acked Read 1 Done Sent Done false
34340 9896 Acked Write 1 Done Sent Done false
34341 9897 Acked Write 7 Done Sent Done false
34342 9898 Acked Read 7 Done Sent Done false
34343 9899 Acked Read 5 Done Sent Done false
34344 9900 Acked Read 7 Done Sent Done false
34345 9901 Acked Read 3 Done Sent Done false
34346 9902 Acked Write 4 Done Sent Done false
34347 9903 Acked Read 6 Done Sent Done false
34348 9904 Acked Read 3 Done Sent Done false
34349 9905 Acked Write 4 Done Sent Done false
34350 9906 Acked Read 1 Done Sent Done false
34351 9907 Acked Write 2 Done Sent Done false
34352 9908 Acked Write 2 Done Sent Done false
34353 9909 Acked Read 1 Done Sent Done false
34354 9910 Acked Write 1 Done Sent Done false
34355 9911 Acked Write 9 Done Sent Done false
34356 9912 Acked Write 9 Done Sent Done false
34357 9913 Acked Flush 0 Done Sent Done false
34358 9914 Acked Read 4 Done Sent Done false
34359 9915 Acked Read 4 Done Sent Done false
34360 9916 Acked Write 6 Done Sent Done false
34361 9917 Acked Read 2 Done Sent Done false
34362 9918 Acked Read 1 Done Sent Done false
34363 9919 Acked Flush 0 Done Sent Done false
34364 9920 Acked Write 3 Done Sent Done false
34365 9921 Acked Read 9 Done Sent Done false
34366 9922 Acked Write 3 Done Sent Done false
34367 9923 Acked Write 8 Done Sent Done false
34368 9924 Acked Write 5 Done Sent Done false
34369 9925 Acked Write 9 Done Sent Done false
34370 9926 Acked Read 10 Done Sent Done false
34371 9927 Acked Read 6 Done Sent Done false
34372 9928 Acked Read 10 Done Sent Done false
34373 9929 Acked Read 3 Done Sent Done false
34374 9930 Acked Write 9 Done Sent Done false
34375 9931 Acked Read 9 Done Sent Done false
34376 9932 Acked Read 5 Done Sent Done false
34377 9933 Acked Read 6 Done Sent Done false
34378 9934 Acked Write 6 Done Sent Done false
34379 9935 Acked Read 1 Done Sent Done false
34380 9936 Acked Read 10 Done Sent Done false
34381 9937 Acked Read 5 Done Sent Done false
34382 9938 Acked Write 4 Done Sent Done false
34383 9939 Acked Read 4 Done Sent Done false
34384 9940 Acked Read 2 Done Sent Done false
34385 9941 Acked Read 6 Done Sent Done false
34386 9942 Acked Write 6 Done Sent Done false
34387 9943 Acked Read 2 Done Sent Done false
34388 9944 Acked Read 8 Done Sent Done false
34389 9945 Acked Read 5 Done Sent Done false
34390 9946 Acked Write 6 Done Sent Done false
34391 9947 Acked Read 8 Done Sent Done false
34392 9948 Acked Flush 0 Done Sent Done false
34393 9949 Acked Write 7 Done Sent Done false
34394 9950 Acked Write 2 Done Sent Done false
34395 9951 Acked Flush 0 Done Sent Done false
34396 9952 Acked Write 1 Done Sent Done false
34397 9953 Acked Read 2 Done Sent Done false
34398 9954 Acked Flush 0 Done Sent Done false
34399 9955 Acked Flush 0 Done Sent Done false
34400 9956 Acked Flush 0 Done Sent Done false
34401 9957 Acked Write 3 Done Sent Done false
34402 9958 Acked Read 5 Done Sent Done false
34403 9959 Acked Read 6 Done Sent Done false
34404 9960 Acked Write 4 Done Sent Done false
34405 9961 Acked Write 3 Done Sent Done false
34406 9962 Acked Read 4 Done Sent Done false
34407 9963 Acked Write 7 Done Sent Done false
34408 9964 Acked Write 5 Done Sent Done false
34409 9965 Acked Write 1 Done Sent Done false
34410 9966 Acked Write 5 Done Sent Done false
34411 9967 Acked Write 5 Done Sent Done false
34412 9968 Acked Write 9 Done Sent Done false
34413 9969 Acked Read 5 Done Sent Done false
34414 9970 Acked Write 10 Done Sent Done false
34415 9971 Acked Flush 0 Done Sent Done false
34416 9972 Acked Write 3 Done Sent Done false
34417 9973 Acked Read 9 Done Sent Done false
34418 9974 Acked Write 2 Done Sent Done false
34419 9975 Acked Read 3 Done Sent Done false
34420 9976 Acked Write 6 Done Sent Done false
34421 9977 Acked Write 6 Done Sent Done false
34422 9978 Acked Write 3 Done Sent Done false
34423 9979 Acked Write 5 Done Sent Done false
34424 9980 Acked Read 5 Done Sent Done false
34425 9981 Acked Read 4 Done Sent Done false
34426 9982 Acked Flush 0 Done Sent Done false
34427 9983 Acked Write 2 Done Sent Done false
34428 9984 Acked Read 10 Done Sent Done false
34429 9985 Acked Write 9 Done Sent Done false
34430 9986 Acked Read 3 Done Sent Done false
34431 9987 Acked Flush 0 Done Sent Done false
34432 9988 Acked Read 4 Done Sent Done false
34433 9989 Acked Write 2 Done Sent Done false
34434 9990 Acked Flush 0 Done Sent Done false
34435 9991 Acked Write 5 Done Sent Done false
34436 9992 Acked Read 2 Done Sent Done false
34437 9993 Acked Read 2 Done Sent Done false
34438 9994 Acked Write 8 Done Sent Done false
34439 9995 Acked Read 7 Done Sent Done false
34440 9996 Acked Read 5 Done Sent Done false
34441 9997 Acked Write 3 Done Sent Done false
34442 9998 Acked Write 7 Done Sent Done false
34443 9999 Acked Read 8 Done Sent Done false
34444 10000 Acked Write 5 Done Sent Done false
34445 10001 Acked Write 3 Done Sent Done false
34446 10002 Acked Read 10 Done Sent Done false
34447 10003 Acked Read 9 Done Sent Done false
34448 10004 Acked Read 2 Done Sent Done false
34449 10005 Acked Read 2 Done Sent Done false
34450 10006 Acked Read 1 Done Sent Done false
34451 10007 Acked Read 6 Done Sent Done false
34452 10008 Acked Read 5 Done Sent Done false
34453 10009 Acked Read 9 Done Sent Done false
34454 10010 Acked Flush 0 Done Sent Done false
34455 10011 Acked Flush 0 Done Sent Done false
34456 10012 Acked Write 4 Done Sent Done false
34457 10013 Acked Read 5 Done Sent Done false
34458 10014 Acked Read 5 Done Sent Done false
34459 10015 Acked Read 5 Done Sent Done false
34460 10016 Acked Read 7 Done Sent Done false
34461 10017 Acked Read 3 Done Sent Done false
34462 10018 Acked Write 6 Done Sent Done false
34463 10019 Acked Read 2 Done Sent Done false
34464 10020 Acked Flush 0 Done Sent Done false
34465 10021 Acked Flush 0 Done Sent Done false
34466 10022 Acked Read 9 Done Sent Done false
34467 10023 Acked Read 8 Done Sent Done false
34468 10024 Acked Flush 0 Done Sent Done false
34469 10025 Acked Flush 0 Done Sent Done false
34470 10026 Acked Write 5 Done Sent Done false
34471 10027 Acked Write 1 Done Sent Done false
34472 10028 Acked Read 8 Done Sent Done false
34473 10029 Acked Flush 0 Done Sent Done false
34474 10030 Acked Read 3 Done Sent Done false
34475 10031 Acked Read 8 Done Sent Done false
34476 10032 Acked Read 2 Done Sent Done false
34477 10033 Acked Write 6 Done Sent Done false
34478 10034 Acked Write 9 Done Sent Done false
34479 10035 Acked Write 8 Done Sent Done false
34480 10036 Acked Write 5 Done Sent Done false
34481 10037 Acked Read 7 Done Sent Done false
34482 10038 Acked Read 7 Done Sent Done false
34483 10039 Acked Write 10 Done Sent Done false
34484 10040 Acked Read 9 Done Sent Done false
34485 10041 Acked Read 3 Done Sent Done false
34486 10042 Acked Read 5 Done Sent Done false
34487 10043 Acked Write 4 Done Sent Done false
34488 10044 Acked Write 1 Done Sent Done false
34489 10045 Acked Write 6 Done Sent Done false
34490 10046 Acked Write 10 Done Sent Done false
34491 10047 Acked Flush 0 Done Sent Done false
34492 10048 Acked Read 3 Done Sent Done false
34493 10049 Acked Read 9 Done Sent Done false
34494 10050 Acked Write 3 Done Sent Done false
34495 10051 Acked Read 3 Done Sent Done false
34496 10052 Acked Read 5 Done Sent Done false
34497 10053 Acked Read 4 Done Sent Done false
34498 10054 Acked Read 5 Done Sent Done false
34499 10055 Acked Write 3 Done Sent Done false
34500 10056 Acked Write 4 Done Sent Done false
34501 10057 Acked Write 5 Done Sent Done false
34502 10058 Acked Read 8 Done Sent Done false
34503 10059 Acked Write 9 Done Sent Done false
34504 10060 Acked Read 4 Done Sent Done false
34505 10061 Acked Write 1 Done Sent Done false
34506 10062 Acked Read 5 Done Sent Done false
34507 10063 Acked Read 9 Done Sent Done false
34508 10064 Acked Flush 0 Done Sent Done false
34509 10065 Acked Write 8 Done Sent Done false
34510 10066 Acked Write 7 Done Sent Done false
34511 10067 Acked Read 6 Done Sent Done false
34512 10068 Acked Write 9 Done Sent Done false
34513 10069 Acked Write 2 Done Sent Done false
34514 10070 Acked Write 8 Done Sent Done false
34515 10071 Acked Flush 0 Done Sent Done false
34516 STATES DS:0 DS:1 DS:2 TOTAL
34517 Sent 0 303 0 303
34518 Done 303 0 303 606
34519 Skipped 0 0 0 0
34520 Error 0 0 0 0
34521 Last Flush: 10071 9768 10071
34522 Downstairs last five completed: 9768 9767 9766 9765 9764
34523 Upstairs last five completed: 10071 10070 10069 10068 10067
34524 CLIENT: Up:0 ds:303 act:2
345252024-11-07T14:10:32.417ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345262024-11-07T14:10:32.417ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49861) = io task client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345272024-11-07T14:10:32.418ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345282024-11-07T14:10:32.418ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345292024-11-07T14:10:32.418ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345302024-11-07T14:10:32.418ZINFOcrucible: send last flush ID to this DS: 9768 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345312024-11-07T14:10:32.419ZINFOcrucible: Replied this last flush ID: 9768 = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345322024-11-07T14:10:32.419ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345332024-11-07T14:10:32.419ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345342024-11-07T14:10:32.419ZINFOcrucible: [1] client re-new 303 jobs since flush 9768 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345352024-11-07T14:10:32.419ZINFOcrucible: [1] Replayed 303 jobs since flush: 9768 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
34536 ----------------------------------------------------------------
34537 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34538 Downstairs last five completed: 10071 10070 10069 10068 10067
34539 Upstairs last five completed: 10071 10070 10069 10068 10067
34540 CLIENT: Up:0 ds:0 act:3
34541 Replay: All jobs finished
34542 Main task has work to do, go find it
34543 got dsc Stop(2)
34544 stop 2
34545 Nov 07 14:10:34.697 INFO request completed, latency_us: 395, response_code: 204, uri: /stop/cid/2, method: POST, req_id: a80eb2a0-52a3-4ae2-b306-115ea96bb0c0, [8830] Got stop action so:false kr:false
34546 remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
345472024-11-07T14:10:34.698ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345482024-11-07T14:10:34.698ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345492024-11-07T14:10:34.698ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345502024-11-07T14:10:34.698ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345512024-11-07T14:10:34.698ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
345522024-11-07T14:10:34.698ZINFOcrucible: client task is exiting = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
34553 Nov 07 14:10:34.698 INFO request completed, latency_us: 154, response_code: 200, uri: /state/cid/2, method: GET, req_id: 4ad854b9-bfb1-4845-9e09-052304498795, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
34554 [8830] Exited with: ExitStatus(unix_wait_status(9))
34555 [8830][2] reports Exit
34556 Nov 07 14:10:38.700 INFO request completed, latency_us: 204, response_code: 200, uri: /state/cid/2, method: GET, req_id: fa02765e-0285-4c8b-9a1d-f47ded7ef802, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
34557 001/300 Read block 3838 len 36864
34558 002/300 Read block 3780 len 24576
34559 003/300 Read block 3935 len 4096
34560 004/300 Read block 3832 len 4096
34561 005/300 Read block 619 len 24576
34562 006/300 Flush
34563 007/300 Read block 1653 len 40960
34564 008/300 Read block 4853 len 32768
34565 009/300 Write block 562 len 16384 data: 7 9 10 7
34566 010/300 Write block 1242 len 32768 data: 4 4 4 4 4 5 5 6
34567 011/300 Read block 453 len 32768
34568 012/300 Read block 2675 len 8192
34569 013/300 Flush
34570 014/300 Read block 1957 len 16384
34571 015/300 Write block 4313 len 12288 data: 10 11 9
34572 016/300 Write block 1628 len 32768 data: 5 4 5 7 4 4 4 5
34573 017/300 Read block 3280 len 12288
34574 018/300 Read block 3415 len 16384
34575 019/300 Write block 2922 len 8192 data: 7 5
34576 020/300 Read block 3141 len 32768
34577 021/300 Write block 1421 len 36864 data: 4 2 3 3 4 5 4 5 6
34578 022/300 Read block 2687 len 12288
34579 023/300 Write block 3304 len 16384 data: 3 7 7 8
34580 024/300 Write block 1402 len 4096 data: 8
34581 025/300 Write block 283 len 24576 data: 4 4 6 6 7 7
34582 026/300 Read block 1942 len 8192
34583 027/300 Write block 4667 len 28672 data: 4 5 7 6 7 8 7
34584 028/300 Write block 47 len 32768 data: 4 5 5 6 5 4 4 5
34585 029/300 Read block 150 len 28672
34586 030/300 Read block 314 len 12288
34587 031/300 Read block 4637 len 40960
34588 032/300 Read block 1336 len 12288
34589 033/300 Read block 225 len 24576
34590 034/300 Write block 2417 len 4096 data: 6
34591 035/300 Read block 641 len 12288
34592 036/300 Write block 1363 len 24576 data: 4 4 5 6 5 5
34593 037/300 Write block 2963 len 40960 data: 5 7 6 5 6 5 7 7 9 9
34594 038/300 Read block 1297 len 4096
34595 039/300 Read block 2100 len 36864
34596 040/300 Read block 2879 len 32768
34597 041/300 Write block 2958 len 20480 data: 6 6 6 5 6
34598 042/300 Read block 1096 len 32768
34599 043/300 Write block 842 len 16384 data: 3 3 3 4
34600 044/300 Flush
34601 045/300 Read block 4457 len 4096
34602 046/300 Write block 3991 len 8192 data: 8 7
34603 047/300 Write block 3779 len 28672 data: 5 5 5 5 5 7 6
34604 048/300 Read block 789 len 4096
34605 049/300 Write block 1467 len 24576 data: 6 7 6 5 5 4
34606 050/300 Write block 1827 len 20480 data: 10 13 12 13 10
34607 051/300 Write block 3383 len 8192 data: 6 6
34608 052/300 Read block 4672 len 32768
34609 053/300 Read block 1265 len 24576
34610 054/300 Read block 221 len 12288
34611 055/300 Write block 764 len 12288 data: 4 4 5
34612 056/300 Read block 3493 len 8192
34613 057/300 Read block 1801 len 32768
34614 058/300 Read block 1891 len 36864
34615 059/300 Write block 4137 len 24576 data: 5 5 5 5 6 4
34616 060/300 Read block 1380 len 32768
34617 061/300 Read block 2816 len 16384
34618 062/300 Write block 3450 len 20480 data: 5 5 3 4 3
34619 063/300 Write block 4299 len 4096 data: 6
34620 064/300 Read block 759 len 36864
34621 065/300 Read block 12 len 24576
34622 066/300 Read block 3848 len 20480
34623 067/300 Read block 1773 len 40960
34624 068/300 Write block 4957 len 32768 data: 10 11 11 11 8 7 7 8
34625 069/300 Read block 3372 len 4096
34626 070/300 Write block 4421 len 8192 data: 7 5
34627 071/300 Read block 1860 len 32768
34628 072/300 Read block 749 len 28672
34629 073/300 Write block 4342 len 20480 data: 6 4 2 5 5
34630 074/300 Read block 3465 len 24576
34631 075/300 Flush
34632 076/300 Write block 2889 len 8192 data: 7 5
34633 077/300 Write block 2807 len 16384 data: 5 5 8 6
34634 078/300 Write block 2994 len 8192 data: 5 6
34635 079/300 Write block 2138 len 4096 data: 6
34636 080/300 Read block 1828 len 8192
34637 081/300 Flush
34638 082/300 Flush
34639 083/300 Read block 4338 len 28672
34640 084/300 Read block 576 len 8192
34641 085/300 Write block 4117 len 40960 data: 5 7 9 8 9 10 9 10 8 8
34642 086/300 Flush
34643 087/300 Write block 3829 len 16384 data: 11 8 7 6
34644 088/300 Write block 4668 len 8192 data: 6 8
34645 089/300 Read block 312 len 20480
34646 090/300 Flush
34647 091/300 Write block 1064 len 20480 data: 3 3 6 4 4
34648 092/300 Read block 4808 len 28672
34649 093/300 Read block 488 len 24576
34650 094/300 Read block 1854 len 40960
34651 095/300 Write block 4464 len 36864 data: 7 8 6 6 6 5 3 3 3
34652 096/300 Read block 3398 len 16384
34653 097/300 Write block 2742 len 24576 data: 6 6 6 6 5 5
34654 098/300 Flush
34655 099/300 Flush
34656 100/300 Write block 3594 len 36864 data: 4 4 4 5 7 8 8 7 6
34657 101/300 Write block 4128 len 36864 data: 6 3 3 3 3 3 4 4 4
34658 102/300 Write block 4428 len 20480 data: 4 4 3 3 4
34659 103/300 Write block 2931 len 16384 data: 4 4 4 3
34660 104/300 Write block 544 len 12288 data: 7 9 7
34661 105/300 Read block 2763 len 8192
34662 106/300 Read block 353 len 24576
34663 107/300 Read block 1366 len 28672
34664 108/300 Write block 4169 len 4096 data: 6
34665 109/300 Read block 349 len 36864
34666 110/300 Read block 817 len 40960
34667 111/300 Read block 4828 len 36864
34668 112/300 Write block 3949 len 36864 data: 4 5 5 6 6 6 7 8 7
34669 113/300 Write block 138 len 40960 data: 6 6 7 7 8 7 5 5 6 7
34670 114/300 Write block 2158 len 12288 data: 7 6 5
34671 115/300 Write block 3190 len 20480 data: 4 5 5 5 6
34672 116/300 Read block 4966 len 4096
34673 117/300 Read block 91 len 8192
34674 118/300 Read block 4407 len 12288
34675 119/300 Flush
34676 120/300 Read block 981 len 16384
34677 121/300 Write block 3849 len 4096 data: 5
34678 122/300 Write block 693 len 24576 data: 5 5 6 6 7 5
34679 123/300 Read block 32 len 12288
34680 124/300 Write block 2491 len 24576 data: 7 5 4 7 8 6
34681 125/300 Write block 3662 len 4096 data: 6
34682 126/300 Read block 4495 len 28672
34683 127/300 Flush
34684 128/300 Write block 1413 len 20480 data: 6 5 5 5 4
34685 129/300 Write block 945 len 20480 data: 3 4 5 4 5
34686 130/300 Read block 4700 len 12288
34687 131/300 Flush
34688 132/300 Flush
34689 133/300 Read block 70 len 16384
34690 134/300 Write block 3498 len 16384 data: 7 6 6 6
34691 135/300 Write block 1459 len 16384 data: 4 4 5 4
34692 136/300 Read block 1803 len 20480
34693 137/300 Read block 2215 len 32768
34694 138/300 Write block 4298 len 36864 data: 6 7 6 6 5 5 6 6 8
34695 139/300 Write block 3790 len 24576 data: 6 7 7 7 4 7
34696 140/300 Read block 2979 len 24576
34697 141/300 Write block 1891 len 16384 data: 6 5 3 2
34698 142/300 Write block 356 len 8192 data: 3 3
34699 143/300 Flush
34700 144/300 Flush
34701 145/300 Read block 2132 len 4096
34702 146/300 Read block 1067 len 8192
34703 147/300 Read block 4144 len 16384
34704 148/300 Read block 1802 len 20480
34705 149/300 Read block 4979 len 4096
34706 150/300 Read block 3504 len 8192
34707 151/300 Write block 2023 len 36864 data: 4 5 5 7 6 7 8 7 6
34708 152/300 Write block 1569 len 36864 data: 6 6 5 6 7 7 9 9 7
34709 153/300 Write block 3592 len 36864 data: 5 4 5 5 5 6 8 9 9
34710 154/300 Write block 4744 len 20480 data: 5 5 6 4 4
34711 155/300 Read block 1333 len 4096
34712 156/300 Read block 4436 len 32768
34713 157/300 Read block 3800 len 16384
34714 158/300 Flush
34715 159/300 Read block 473 len 32768
34716 160/300 Write block 2762 len 28672 data: 6 6 5 4 4 5 5
34717 161/300 Read block 3752 len 32768
34718 162/300 Read block 1955 len 40960
34719 163/300 Read block 3719 len 36864
34720 164/300 Write block 1430 len 12288 data: 6 7 8
34721 165/300 Read block 180 len 4096
34722 166/300 Read block 4471 len 36864
34723 167/300 Write block 2380 len 24576 data: 5 4 5 5 6 4
34724 168/300 Flush
34725 169/300 Flush
34726 170/300 Flush
34727 171/300 Read block 1636 len 36864
34728 172/300 Write block 4835 len 16384 data: 3 5 6 6
34729 173/300 Read block 4099 len 12288
34730 174/300 Read block 540 len 16384
34731 175/300 Flush
34732 176/300 Read block 2487 len 32768
34733 177/300 Read block 3038 len 20480
34734 178/300 Read block 1354 len 8192
34735 179/300 Read block 3469 len 32768
34736 180/300 Read block 1925 len 28672
34737 181/300 Flush
34738 182/300 Write block 2091 len 4096 data: 5
34739 183/300 Write block 3102 len 40960 data: 5 5 5 8 6 6 6 5 5 5
34740 184/300 Read block 1389 len 36864
34741 185/300 Read block 1651 len 40960
34742 186/300 Read block 1699 len 8192
34743 187/300 Write block 2931 len 16384 data: 5 5 5 4
34744 188/300 Write block 2065 len 40960 data: 5 3 3 4 4 3 4 3 3 4
34745 189/300 Read block 4412 len 40960
34746 190/300 Write block 2569 len 36864 data: 5 4 6 5 4 4 6 7 7
34747 191/300 Read block 4841 len 4096
34748 192/300 Read block 4879 len 36864
34749 193/300 Write block 4359 len 36864 data: 5 5 5 7 6 5 5 7 10
34750 194/300 Write block 1815 len 36864 data: 3 4 5 4 4 4 5 6 8
34751 195/300 Write block 760 len 40960 data: 6 5 7 6 5 5 6 6 7 8
34752 196/300 Write block 922 len 32768 data: 5 3 3 3 3 4 5 3
34753 197/300 Read block 3657 len 16384
34754 198/300 Read block 2683 len 8192
34755 199/300 Read block 318 len 36864
34756 200/300 Write block 1495 len 36864 data: 9 9 7 7 6 4 4 5 5
34757 201/300 Read block 463 len 40960
34758 202/300 Write block 155 len 40960 data: 5 5 7 7 9 10 10 8 8 6
34759 203/300 Read block 547 len 36864
34760 204/300 Write block 1755 len 24576 data: 9 8 7 6 8 7
34761 205/300 Write block 2266 len 24576 data: 4 4 5 6 7 7
34762 206/300 Write block 3595 len 12288 data: 6 6 7
34763 207/300 Read block 4859 len 24576
34764 208/300 Write block 306 len 20480 data: 7 10 9 8 7
34765 209/300 Read block 3887 len 24576
34766 210/300 Read block 4229 len 24576
34767 211/300 Write block 403 len 4096 data: 8
34768 212/300 Write block 2264 len 32768 data: 4 4 5 5 6 7 8 8
34769 213/300 Read block 1731 len 20480
34770 214/300 Write block 2542 len 20480 data: 11 11 9 8 4
34771 215/300 Read block 3000 len 20480
34772 216/300 Write block 1486 len 20480 data: 9 7 7 7 7
34773 217/300 Write block 3042 len 32768 data: 5 4 4 3 4 5 6 6
34774 218/300 Write block 2918 len 40960 data: 9 9 8 8 8 6 5 4 4 4
34775 219/300 Write block 136 len 24576 data: 6 6 7 7 8 8
34776 220/300 Flush
34777 221/300 Read block 3331 len 20480
34778 222/300 Read block 72 len 32768
34779 223/300 Read block 3392 len 32768
34780 224/300 Read block 3916 len 4096
34781 225/300 Write block 105 len 36864 data: 2 2 2 3 3 3 4 4 3
34782 226/300 Write block 2838 len 24576 data: 3 2 2 2 2 2
34783 227/300 Write block 1654 len 36864 data: 10 9 9 9 7 9 10 10 7
34784 228/300 Write block 188 len 24576 data: 10 10 10 8 8 7
34785 229/300 Read block 2708 len 40960
34786 230/300 Read block 2847 len 32768
34787 231/300 Read block 3014 len 16384
34788 232/300 Read block 2177 len 32768
34789 233/300 Read block 1857 len 12288
34790 234/300 Write block 3072 len 8192 data: 4 5
34791 235/300 Flush
34792 236/300 Read block 2728 len 12288
34793 237/300 Read block 1357 len 4096
34794 238/300 Write block 1748 len 4096 data: 5
34795 239/300 Write block 2731 len 12288 data: 8 11 9
34796 240/300 Write block 3210 len 20480 data: 6 6 3 6 6
34797 241/300 Write block 4650 len 16384 data: 8 8 8 6
34798 242/300 Read block 2551 len 24576
34799 243/300 Write block 1219 len 8192 data: 6 6
34800 244/300 Write block 1741 len 20480 data: 7 7 6 5 6
34801 245/300 Read block 3443 len 4096
34802 246/300 Read block 2571 len 32768
34803 247/300 Write block 3173 len 8192 data: 4 4
34804 248/300 Write block 2575 len 8192 data: 7 8
34805 249/300 Read block 2383 len 32768
34806 250/300 Write block 1380 len 24576 data: 7 6 5 5 4 7
34807 251/300 Read block 4317 len 36864
34808 252/300 Read block 4451 len 32768
34809 253/300 Read block 3936 len 16384
34810 254/300 Write block 4671 len 12288 data: 8 9 8
34811 255/300 Write block 2937 len 4096 data: 8
34812 256/300 Read block 3879 len 4096
34813 257/300 Write block 2148 len 20480 data: 5 6 9 7 7
34814 258/300 Read block 1116 len 4096
34815 259/300 Read block 1019 len 16384
34816 260/300 Flush
34817 261/300 Read block 926 len 28672
34818 262/300 Read block 4923 len 36864
34819 263/300 Read block 920 len 8192
34820 264/300 Write block 1034 len 36864 data: 3 3 4 4 4 4 4 5 5
34821 265/300 Flush
34822 266/300 Write block 4708 len 4096 data: 6
34823 267/300 Read block 4413 len 28672
34824 268/300 Read block 745 len 28672
34825 269/300 Read block 142 len 36864
34826 270/300 Read block 3178 len 4096
34827 271/300 Write block 4688 len 24576 data: 5 4 5 4 4 5
34828 272/300 Write block 1690 len 16384 data: 4 3 3 3
34829 273/300 Read block 3132 len 8192
34830 274/300 Write block 3042 len 8192 data: 6 5
34831 275/300 Read block 333 len 36864
34832 276/300 Flush
34833 277/300 Read block 3764 len 16384
34834 278/300 Read block 2911 len 28672
34835 279/300 Read block 3325 len 40960
34836 280/300 Read block 1271 len 12288
34837 281/300 Write block 380 len 4096 data: 10
34838 282/300 Read block 4370 len 32768
34839 283/300 Read block 2915 len 8192
34840 284/300 Read block 2905 len 24576
34841 285/300 Read block 4990 len 20480
34842 286/300 Read block 97 len 16384
34843 287/300 Write block 4277 len 32768 data: 8 8 6 7 7 5 5 5
34844 288/300 Write block 3509 len 40960 data: 9 8 9 9 8 7 5 5 4 4
34845 289/300 Write block 2092 len 16384 data: 6 5 4 3
34846 290/300 Read block 2343 len 16384
34847 291/300 Read block 598 len 32768
34848 292/300 Write block 1747 len 32768 data: 5 6 9 12 13 11 11 11
34849 293/300 Read block 2070 len 24576
34850 294/300 Write block 3447 len 20480 data: 5 5 4 6 6
34851 295/300 Write block 4099 len 36864 data: 5 6 6 7 8 8 9 8 5
34852 296/300 Write block 3801 len 8192 data: 8 7
34853 297/300 Write block 4855 len 20480 data: 2 2 2 2 3
34854 298/300 Flush
34855 299/300 Write block 4052 len 32768 data: 5 5 4 4 3 3 4 4
34856 300/300 Read block 3443 len 32768
34857 301/300 Write block 3568 len 32768 data: 7 9 7 6 5 5 5 5
34858 Main task has work to do, go find it
34859 got dsc Start(2)
34860 start 2
34861 Nov 07 14:10:38.977 INFO request completed, latency_us: 236, response_code: 204, uri: /start/cid/2, method: POST, req_id: aa3df91c-73a5-4b12-94ae-be109d9ba74e, remote_addr: 127.0.0.1:34100, local_addr: 127.0.0.1:9998
34862 Replay: started 2, returned:Ok(())
34863 ----------------------------------------------------------------
34864 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
34865 States: Active Active Offline
34866 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34867 10072 Acked Read 9 Done Done Sent false
34868 10073 Acked Read 6 Done Done Sent false
34869 10074 Acked Read 1 Done Done Sent false
34870 10075 Acked Read 1 Done Done Sent false
34871 10076 Acked Read 6 Done Done Sent false
34872 10077 Acked Flush 0 Done Done Sent false
34873 10078 Acked Read 10 Done Done Sent false
34874 10079 Acked Read 8 Done Done Sent false
34875 10080 Acked Write 4 Done Done Sent false
34876 10081 Acked Write 8 Done Done Sent false
34877 10082 Acked Read 8 Done Done Sent false
34878 10083 Acked Read 2 Done Done Sent false
34879 10084 Acked Flush 0 Done Done Sent false
34880 10085 Acked Read 4 Done Done Sent false
34881 10086 Acked Write 3 Done Done Sent false
34882 10087 Acked Write 8 Done Done Sent false
34883 10088 Acked Read 3 Done Done Sent false
34884 10089 Acked Read 4 Done Done Sent false
34885 10090 Acked Write 2 Done Done Sent false
34886 10091 Acked Read 8 Done Done Sent false
34887 10092 Acked Write 9 Done Done Sent false
34888 10093 Acked Read 3 Done Done Sent false
34889 10094 Acked Write 4 Done Done Sent false
34890 10095 Acked Write 1 Done Done Sent false
34891 10096 Acked Write 6 Done Done Sent false
34892 10097 Acked Read 2 Done Done Sent false
34893 10098 Acked Write 7 Done Done Sent false
34894 10099 Acked Write 8 Done Done Sent false
34895 10100 Acked Read 7 Done Done Sent false
34896 10101 Acked Read 3 Done Done Sent false
34897 10102 Acked Read 10 Done Done Sent false
34898 10103 Acked Read 3 Done Done Sent false
34899 10104 Acked Read 6 Done Done Sent false
34900 10105 Acked Write 1 Done Done Sent false
34901 10106 Acked Read 3 Done Done Sent false
34902 10107 Acked Write 6 Done Done Sent false
34903 10108 Acked Write 10 Done Done Sent false
34904 10109 Acked Read 1 Done Done Sent false
34905 10110 Acked Read 9 Done Done Sent false
34906 10111 Acked Read 8 Done Done Sent false
34907 10112 Acked Write 5 Done Done Sent false
34908 10113 Acked Read 8 Done Done Sent false
34909 10114 Acked Write 4 Done Done Sent false
34910 10115 Acked Flush 0 Done Done Sent false
34911 10116 Acked Read 1 Done Done Sent false
34912 10117 Acked Write 2 Done Done Sent false
34913 10118 Acked Write 7 Done Done Sent false
34914 10119 Acked Read 1 Done Done Sent false
34915 10120 Acked Write 6 Done Done Sent false
34916 10121 Acked Write 5 Done Done Sent false
34917 10122 Acked Write 2 Done Done Sent false
34918 10123 Acked Read 8 Done Done Sent false
34919 10124 Acked Read 6 Done Done Sent false
34920 10125 Acked Read 3 Done Done Sent false
34921 10126 Acked Write 3 Done Done Sent false
34922 10127 Acked Read 2 Done Done Sent false
34923 10128 Acked Read 8 Done Done Sent false
34924 10129 Acked Read 9 Done Done Sent false
34925 10130 Acked Write 6 Done Done Sent false
34926 10131 Acked Read 8 Done Done Sent false
34927 10132 Acked Read 4 Done Done Sent false
34928 10133 Acked Write 5 Done Done Sent false
34929 10134 Acked Write 1 Done Done Sent false
34930 10135 Acked Read 9 Done Done Sent false
34931 10136 Acked Read 6 Done Done Sent false
34932 10137 Acked Read 5 Done Done Sent false
34933 10138 Acked Read 10 Done Done Sent false
34934 10139 Acked Write 8 Done Done Sent false
34935 10140 Acked Read 1 Done Done Sent false
34936 10141 Acked Write 2 Done Done Sent false
34937 10142 Acked Read 8 Done Done Sent false
34938 10143 Acked Read 7 Done Done Sent false
34939 10144 Acked Write 5 Done Done Sent false
34940 10145 Acked Read 6 Done Done Sent false
34941 10146 Acked Flush 0 Done Done Sent false
34942 10147 Acked Write 2 Done Done Sent false
34943 10148 Acked Write 4 Done Done Sent false
34944 10149 Acked Write 2 Done Done Sent false
34945 10150 Acked Write 1 Done Done Sent false
34946 10151 Acked Read 2 Done Done Sent false
34947 10152 Acked Flush 0 Done Done Sent false
34948 10153 Acked Flush 0 Done Done Sent false
34949 10154 Acked Read 7 Done Done Sent false
34950 10155 Acked Read 2 Done Done Sent false
34951 10156 Acked Write 10 Done Done Sent false
34952 10157 Acked Flush 0 Done Done Sent false
34953 10158 Acked Write 4 Done Done Sent false
34954 10159 Acked Write 2 Done Done Sent false
34955 10160 Acked Read 5 Done Done Sent false
34956 10161 Acked Flush 0 Done Done Sent false
34957 10162 Acked Write 5 Done Done Sent false
34958 10163 Acked Read 7 Done Done Sent false
34959 10164 Acked Read 6 Done Done Sent false
34960 10165 Acked Read 10 Done Done Sent false
34961 10166 Acked Write 9 Done Done Sent false
34962 10167 Acked Read 4 Done Done Sent false
34963 10168 Acked Write 6 Done Done Sent false
34964 10169 Acked Flush 0 Done Done Sent false
34965 10170 Acked Flush 0 Done Done Sent false
34966 10171 Acked Write 9 Done Done Sent false
34967 10172 Acked Write 9 Done Done Sent false
34968 10173 Acked Write 5 Done Done Sent false
34969 10174 Acked Write 4 Done Done Sent false
34970 10175 Acked Write 3 Done Done Sent false
34971 10176 Acked Read 2 Done Done Sent false
34972 10177 Acked Read 6 Done Done Sent false
34973 10178 Acked Read 7 Done Done Sent false
34974 10179 Acked Write 1 Done Done Sent false
34975 10180 Acked Read 9 Done Done Sent false
34976 10181 Acked Read 10 Done Done Sent false
34977 10182 Acked Read 9 Done Done Sent false
34978 10183 Acked Write 9 Done Done Sent false
34979 10184 Acked Write 10 Done Done Sent false
34980 10185 Acked Write 3 Done Done Sent false
34981 10186 Acked Write 5 Done Done Sent false
34982 10187 Acked Read 1 Done Done Sent false
34983 10188 Acked Read 2 Done Done Sent false
34984 10189 Acked Read 3 Done Done Sent false
34985 10190 Acked Flush 0 Done Done Sent false
34986 10191 Acked Read 4 Done Done Sent false
34987 10192 Acked Write 1 Done Done Sent false
34988 10193 Acked Write 6 Done Done Sent false
34989 10194 Acked Read 3 Done Done Sent false
34990 10195 Acked Write 6 Done Done Sent false
34991 10196 Acked Write 1 Done Done Sent false
34992 10197 Acked Read 7 Done Done Sent false
34993 10198 Acked Flush 0 Done Done Sent false
34994 10199 Acked Write 5 Done Done Sent false
34995 10200 Acked Write 5 Done Done Sent false
34996 10201 Acked Read 3 Done Done Sent false
34997 10202 Acked Flush 0 Done Done Sent false
34998 10203 Acked Flush 0 Done Done Sent false
34999 10204 Acked Read 4 Done Done Sent false
35000 10205 Acked Write 4 Done Done Sent false
35001 10206 Acked Write 4 Done Done Sent false
35002 10207 Acked Read 5 Done Done Sent false
35003 10208 Acked Read 8 Done Done Sent false
35004 10209 Acked Write 9 Done Done Sent false
35005 10210 Acked Write 6 Done Done Sent false
35006 10211 Acked Read 6 Done Done Sent false
35007 10212 Acked Write 4 Done Done Sent false
35008 10213 Acked Write 2 Done Done Sent false
35009 10214 Acked Flush 0 Done Done Sent false
35010 10215 Acked Flush 0 Done Done Sent false
35011 10216 Acked Read 1 Done Done Sent false
35012 10217 Acked Read 2 Done Done Sent false
35013 10218 Acked Read 4 Done Done Sent false
35014 10219 Acked Read 5 Done Done Sent false
35015 10220 Acked Read 1 Done Done Sent false
35016 10221 Acked Read 2 Done Done Sent false
35017 10222 Acked Write 9 Done Done Sent false
35018 10223 Acked Write 9 Done Done Sent false
35019 10224 Acked Write 9 Done Done Sent false
35020 10225 Acked Write 5 Done Done Sent false
35021 10226 Acked Read 1 Done Done Sent false
35022 10227 Acked Read 8 Done Done Sent false
35023 10228 Acked Read 4 Done Done Sent false
35024 10229 Acked Flush 0 Done Done Sent false
35025 10230 Acked Read 8 Done Done Sent false
35026 10231 Acked Write 7 Done Done Sent false
35027 10232 Acked Read 8 Done Done Sent false
35028 10233 Acked Read 10 Done Done Sent false
35029 10234 Acked Read 9 Done Done Sent false
35030 10235 Acked Write 3 Done Done Sent false
35031 10236 Acked Read 1 Done Done Sent false
35032 10237 Acked Read 9 Done Done Sent false
35033 10238 Acked Write 6 Done Done Sent false
35034 10239 Acked Flush 0 Done Done Sent false
35035 10240 Acked Flush 0 Done Done Sent false
35036 10241 Acked Flush 0 Done Done Sent false
35037 10242 Acked Read 9 Done Done Sent false
35038 10243 Acked Write 4 Done Done Sent false
35039 10244 Acked Read 3 Done Done Sent false
35040 10245 Acked Read 4 Done Done Sent false
35041 10246 Acked Flush 0 Done Done Sent false
35042 10247 Acked Read 8 Done Done Sent false
35043 10248 Acked Read 5 Done Done Sent false
35044 10249 Acked Read 2 Done Done Sent false
35045 10250 Acked Read 8 Done Done Sent false
35046 10251 Acked Read 7 Done Done Sent false
35047 10252 Acked Flush 0 Done Done Sent false
35048 10253 Acked Write 1 Done Done Sent false
35049 10254 Acked Write 10 Done Done Sent false
35050 10255 Acked Read 9 Done Done Sent false
35051 10256 Acked Read 10 Done Done Sent false
35052 10257 Acked Read 2 Done Done Sent false
35053 10258 Acked Write 4 Done Done Sent false
35054 10259 Acked Write 10 Done Done Sent false
35055 10260 Acked Read 10 Done Done Sent false
35056 10261 Acked Write 9 Done Done Sent false
35057 10262 Acked Read 1 Done Done Sent false
35058 10263 Acked Read 9 Done Done Sent false
35059 10264 Acked Write 9 Done Done Sent false
35060 10265 Acked Write 9 Done Done Sent false
35061 10266 Acked Write 10 Done Done Sent false
35062 10267 Acked Write 8 Done Done Sent false
35063 10268 Acked Read 4 Done Done Sent false
35064 10269 Acked Read 2 Done Done Sent false
35065 10270 Acked Read 9 Done Done Sent false
35066 10271 Acked Write 9 Done Done Sent false
35067 10272 Acked Read 10 Done Done Sent false
35068 10273 Acked Write 10 Done Done Sent false
35069 10274 Acked Read 9 Done Done Sent false
35070 10275 Acked Write 6 Done Done Sent false
35071 10276 Acked Write 6 Done Done Sent false
35072 10277 Acked Write 3 Done Done Sent false
35073 10278 Acked Read 6 Done Done Sent false
35074 10279 Acked Write 5 Done Done Sent false
35075 10280 Acked Read 6 Done Done Sent false
35076 10281 Acked Read 6 Done Done Sent false
35077 10282 Acked Write 1 Done Done Sent false
35078 10283 Acked Write 8 Done Done Sent false
35079 10284 Acked Read 5 Done Done Sent false
35080 10285 Acked Write 5 Done Done Sent false
35081 10286 Acked Read 5 Done Done Sent false
35082 10287 Acked Write 5 Done Done Sent false
35083 10288 Acked Write 8 Done Done Sent false
35084 10289 Acked Write 10 Done Done Sent false
35085 10290 Acked Write 6 Done Done Sent false
35086 10291 Acked Flush 0 Done Done Sent false
35087 10292 Acked Read 5 Done Done Sent false
35088 10293 Acked Read 8 Done Done Sent false
35089 10294 Acked Read 8 Done Done Sent false
35090 10295 Acked Read 1 Done Done Sent false
35091 10296 Acked Write 9 Done Done Sent false
35092 10297 Acked Write 6 Done Done Sent false
35093 10298 Acked Write 9 Done Done Sent false
35094 10299 Acked Write 6 Done Done Sent false
35095 10300 Acked Read 10 Done Done Sent false
35096 10301 Acked Read 8 Done Done Sent false
35097 10302 Acked Read 4 Done Done Sent false
35098 10303 Acked Read 8 Done Done Sent false
35099 10304 Acked Read 3 Done Done Sent false
35100 10305 Acked Write 2 Done Done Sent false
35101 10306 Acked Flush 0 Done Done Sent false
35102 10307 Acked Read 3 Done Done Sent false
35103 10308 Acked Read 1 Done Done Sent false
35104 10309 Acked Write 1 Done Done Sent false
35105 10310 Acked Write 3 Done Done Sent false
35106 10311 Acked Write 5 Done Done Sent false
35107 10312 Acked Write 4 Done Done Sent false
35108 10313 Acked Read 6 Done Done Sent false
35109 10314 Acked Write 2 Done Done Sent false
35110 10315 Acked Write 5 Done Done Sent false
35111 10316 Acked Read 1 Done Done Sent false
35112 10317 Acked Read 8 Done Done Sent false
35113 10318 Acked Write 2 Done Done Sent false
35114 10319 Acked Write 2 Done Done Sent false
35115 10320 Acked Read 8 Done Done Sent false
35116 10321 Acked Write 6 Done Done Sent false
35117 10322 Acked Read 9 Done Done Sent false
35118 10323 Acked Read 8 Done Done Sent false
35119 10324 Acked Read 4 Done Done Sent false
35120 10325 Acked Write 3 Done Done Sent false
35121 10326 Acked Write 1 Done Done Sent false
35122 10327 Acked Read 1 Done Done Sent false
35123 10328 Acked Write 5 Done Done Sent false
35124 10329 Acked Read 1 Done Done Sent false
35125 10330 Acked Read 4 Done Done Sent false
35126 10331 Acked Flush 0 Done Done Sent false
35127 10332 Acked Read 7 Done Done Sent false
35128 10333 Acked Read 9 Done Done Sent false
35129 10334 Acked Read 2 Done Done Sent false
35130 10335 Acked Write 9 Done Done Sent false
35131 10336 Acked Flush 0 Done Done Sent false
35132 10337 Acked Write 1 Done Done Sent false
35133 10338 Acked Read 7 Done Done Sent false
35134 10339 Acked Read 7 Done Done Sent false
35135 10340 Acked Read 9 Done Done Sent false
35136 10341 Acked Read 1 Done Done Sent false
35137 10342 Acked Write 6 Done Done Sent false
35138 10343 Acked Write 4 Done Done Sent false
35139 10344 Acked Read 2 Done Done Sent false
35140 10345 Acked Write 2 Done Done Sent false
35141 10346 Acked Read 9 Done Done Sent false
35142 10347 Acked Flush 0 Done Done Sent false
35143 10348 Acked Read 4 Done Done Sent false
35144 10349 Acked Read 7 Done Done Sent false
35145 10350 Acked Read 10 Done Done Sent false
35146 10351 Acked Read 3 Done Done Sent false
35147 10352 Acked Write 1 Done Done Sent false
35148 10353 Acked Read 8 Done Done Sent false
35149 10354 Acked Read 2 Done Done Sent false
35150 10355 Acked Read 6 Done Done Sent false
35151 10356 Acked Read 5 Done Done Sent false
35152 10357 Acked Read 4 Done Done Sent false
35153 10358 Acked Write 8 Done Done Sent false
35154 10359 Acked Write 10 Done Done Sent false
35155 10360 Acked Write 4 Done Done Sent false
35156 10361 Acked Read 4 Done Done Sent false
35157 10362 Acked Read 8 Done Done Sent false
35158 10363 Acked Write 8 Done Done Sent false
35159 10364 Acked Read 6 Done Done Sent false
35160 10365 Acked Write 5 Done Done Sent false
35161 10366 Acked Write 9 Done Done Sent false
35162 10367 Acked Write 2 Done Done Sent false
35163 10368 Acked Write 5 Done Done Sent false
35164 10369 Acked Flush 0 Done Done Sent false
35165 10370 Acked Write 8 Done Done Sent false
35166 10371 Acked Read 8 Done Done Sent false
35167 10372 Acked Write 8 Done Done Sent false
35168 STATES DS:0 DS:1 DS:2 TOTAL
35169 Sent 0 0 301 301
35170 Done 301 301 0 602
35171 Skipped 0 0 0 0
35172 Error 0 0 0 0
35173 Last Flush: 10369 10369 10071
35174 Downstairs last five completed: 10071 10070 10069 10068 10067
35175 Upstairs last five completed: 10372 10371 10370 10369 10368
35176 CLIENT: Up:0 ds:301 act:2
35177 [8830] got start action
35178 [8830] I am going to restart
35179 Starting downstairs at port 8830
35180 Make output file at "/tmp/dsc/downstairs-8830.txt"
35181 [8830][2] reports Starting
35182 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1230)
35183 [8830][2] reports Running
35184 ----------------------------------------------------------------
35185 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
35186 States: Active Active Offline
35187 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35188 10072 Acked Read 9 Done Done Sent false
35189 10073 Acked Read 6 Done Done Sent false
35190 10074 Acked Read 1 Done Done Sent false
35191 10075 Acked Read 1 Done Done Sent false
35192 10076 Acked Read 6 Done Done Sent false
35193 10077 Acked Flush 0 Done Done Sent false
35194 10078 Acked Read 10 Done Done Sent false
35195 10079 Acked Read 8 Done Done Sent false
35196 10080 Acked Write 4 Done Done Sent false
35197 10081 Acked Write 8 Done Done Sent false
35198 10082 Acked Read 8 Done Done Sent false
35199 10083 Acked Read 2 Done Done Sent false
35200 10084 Acked Flush 0 Done Done Sent false
35201 10085 Acked Read 4 Done Done Sent false
35202 10086 Acked Write 3 Done Done Sent false
35203 10087 Acked Write 8 Done Done Sent false
35204 10088 Acked Read 3 Done Done Sent false
35205 10089 Acked Read 4 Done Done Sent false
35206 10090 Acked Write 2 Done Done Sent false
35207 10091 Acked Read 8 Done Done Sent false
35208 10092 Acked Write 9 Done Done Sent false
35209 10093 Acked Read 3 Done Done Sent false
35210 10094 Acked Write 4 Done Done Sent false
35211 10095 Acked Write 1 Done Done Sent false
35212 10096 Acked Write 6 Done Done Sent false
35213 10097 Acked Read 2 Done Done Sent false
35214 10098 Acked Write 7 Done Done Sent false
35215 10099 Acked Write 8 Done Done Sent false
35216 10100 Acked Read 7 Done Done Sent false
35217 10101 Acked Read 3 Done Done Sent false
35218 10102 Acked Read 10 Done Done Sent false
35219 10103 Acked Read 3 Done Done Sent false
35220 10104 Acked Read 6 Done Done Sent false
35221 10105 Acked Write 1 Done Done Sent false
35222 10106 Acked Read 3 Done Done Sent false
35223 10107 Acked Write 6 Done Done Sent false
35224 10108 Acked Write 10 Done Done Sent false
35225 10109 Acked Read 1 Done Done Sent false
35226 10110 Acked Read 9 Done Done Sent false
35227 10111 Acked Read 8 Done Done Sent false
35228 10112 Acked Write 5 Done Done Sent false
35229 10113 Acked Read 8 Done Done Sent false
35230 10114 Acked Write 4 Done Done Sent false
35231 10115 Acked Flush 0 Done Done Sent false
35232 10116 Acked Read 1 Done Done Sent false
35233 10117 Acked Write 2 Done Done Sent false
35234 10118 Acked Write 7 Done Done Sent false
35235 10119 Acked Read 1 Done Done Sent false
35236 10120 Acked Write 6 Done Done Sent false
35237 10121 Acked Write 5 Done Done Sent false
35238 10122 Acked Write 2 Done Done Sent false
35239 10123 Acked Read 8 Done Done Sent false
35240 10124 Acked Read 6 Done Done Sent false
35241 10125 Acked Read 3 Done Done Sent false
35242 10126 Acked Write 3 Done Done Sent false
35243 10127 Acked Read 2 Done Done Sent false
35244 10128 Acked Read 8 Done Done Sent false
35245 10129 Acked Read 9 Done Done Sent false
35246 10130 Acked Write 6 Done Done Sent false
35247 10131 Acked Read 8 Done Done Sent false
35248 10132 Acked Read 4 Done Done Sent false
35249 10133 Acked Write 5 Done Done Sent false
35250 10134 Acked Write 1 Done Done Sent false
35251 10135 Acked Read 9 Done Done Sent false
35252 10136 Acked Read 6 Done Done Sent false
35253 10137 Acked Read 5 Done Done Sent false
35254 10138 Acked Read 10 Done Done Sent false
35255 10139 Acked Write 8 Done Done Sent false
35256 10140 Acked Read 1 Done Done Sent false
35257 10141 Acked Write 2 Done Done Sent false
35258 10142 Acked Read 8 Done Done Sent false
35259 10143 Acked Read 7 Done Done Sent false
35260 10144 Acked Write 5 Done Done Sent false
35261 10145 Acked Read 6 Done Done Sent false
35262 10146 Acked Flush 0 Done Done Sent false
35263 10147 Acked Write 2 Done Done Sent false
35264 10148 Acked Write 4 Done Done Sent false
35265 10149 Acked Write 2 Done Done Sent false
35266 10150 Acked Write 1 Done Done Sent false
35267 10151 Acked Read 2 Done Done Sent false
35268 10152 Acked Flush 0 Done Done Sent false
35269 10153 Acked Flush 0 Done Done Sent false
35270 10154 Acked Read 7 Done Done Sent false
35271 10155 Acked Read 2 Done Done Sent false
35272 10156 Acked Write 10 Done Done Sent false
35273 10157 Acked Flush 0 Done Done Sent false
35274 10158 Acked Write 4 Done Done Sent false
35275 10159 Acked Write 2 Done Done Sent false
35276 10160 Acked Read 5 Done Done Sent false
35277 10161 Acked Flush 0 Done Done Sent false
35278 10162 Acked Write 5 Done Done Sent false
35279 10163 Acked Read 7 Done Done Sent false
35280 10164 Acked Read 6 Done Done Sent false
35281 10165 Acked Read 10 Done Done Sent false
35282 10166 Acked Write 9 Done Done Sent false
35283 10167 Acked Read 4 Done Done Sent false
35284 10168 Acked Write 6 Done Done Sent false
35285 10169 Acked Flush 0 Done Done Sent false
35286 10170 Acked Flush 0 Done Done Sent false
35287 10171 Acked Write 9 Done Done Sent false
35288 10172 Acked Write 9 Done Done Sent false
35289 10173 Acked Write 5 Done Done Sent false
35290 10174 Acked Write 4 Done Done Sent false
35291 10175 Acked Write 3 Done Done Sent false
35292 10176 Acked Read 2 Done Done Sent false
35293 10177 Acked Read 6 Done Done Sent false
35294 10178 Acked Read 7 Done Done Sent false
35295 10179 Acked Write 1 Done Done Sent false
35296 10180 Acked Read 9 Done Done Sent false
35297 10181 Acked Read 10 Done Done Sent false
35298 10182 Acked Read 9 Done Done Sent false
35299 10183 Acked Write 9 Done Done Sent false
35300 10184 Acked Write 10 Done Done Sent false
35301 10185 Acked Write 3 Done Done Sent false
35302 10186 Acked Write 5 Done Done Sent false
35303 10187 Acked Read 1 Done Done Sent false
35304 10188 Acked Read 2 Done Done Sent false
35305 10189 Acked Read 3 Done Done Sent false
35306 10190 Acked Flush 0 Done Done Sent false
35307 10191 Acked Read 4 Done Done Sent false
35308 10192 Acked Write 1 Done Done Sent false
35309 10193 Acked Write 6 Done Done Sent false
35310 10194 Acked Read 3 Done Done Sent false
35311 10195 Acked Write 6 Done Done Sent false
35312 10196 Acked Write 1 Done Done Sent false
35313 10197 Acked Read 7 Done Done Sent false
35314 10198 Acked Flush 0 Done Done Sent false
35315 10199 Acked Write 5 Done Done Sent false
35316 10200 Acked Write 5 Done Done Sent false
35317 10201 Acked Read 3 Done Done Sent false
35318 10202 Acked Flush 0 Done Done Sent false
35319 10203 Acked Flush 0 Done Done Sent false
35320 10204 Acked Read 4 Done Done Sent false
35321 10205 Acked Write 4 Done Done Sent false
35322 10206 Acked Write 4 Done Done Sent false
35323 10207 Acked Read 5 Done Done Sent false
35324 10208 Acked Read 8 Done Done Sent false
35325 10209 Acked Write 9 Done Done Sent false
35326 10210 Acked Write 6 Done Done Sent false
35327 10211 Acked Read 6 Done Done Sent false
35328 10212 Acked Write 4 Done Done Sent false
35329 10213 Acked Write 2 Done Done Sent false
35330 10214 Acked Flush 0 Done Done Sent false
35331 10215 Acked Flush 0 Done Done Sent false
35332 10216 Acked Read 1 Done Done Sent false
35333 10217 Acked Read 2 Done Done Sent false
35334 10218 Acked Read 4 Done Done Sent false
35335 10219 Acked Read 5 Done Done Sent false
35336 10220 Acked Read 1 Done Done Sent false
35337 10221 Acked Read 2 Done Done Sent false
35338 10222 Acked Write 9 Done Done Sent false
35339 10223 Acked Write 9 Done Done Sent false
35340 10224 Acked Write 9 Done Done Sent false
35341 10225 Acked Write 5 Done Done Sent false
35342 10226 Acked Read 1 Done Done Sent false
35343 10227 Acked Read 8 Done Done Sent false
35344 10228 Acked Read 4 Done Done Sent false
35345 10229 Acked Flush 0 Done Done Sent false
35346 10230 Acked Read 8 Done Done Sent false
35347 10231 Acked Write 7 Done Done Sent false
35348 10232 Acked Read 8 Done Done Sent false
35349 10233 Acked Read 10 Done Done Sent false
35350 10234 Acked Read 9 Done Done Sent false
35351 10235 Acked Write 3 Done Done Sent false
35352 10236 Acked Read 1 Done Done Sent false
35353 10237 Acked Read 9 Done Done Sent false
35354 10238 Acked Write 6 Done Done Sent false
35355 10239 Acked Flush 0 Done Done Sent false
35356 10240 Acked Flush 0 Done Done Sent false
35357 10241 Acked Flush 0 Done Done Sent false
35358 10242 Acked Read 9 Done Done Sent false
35359 10243 Acked Write 4 Done Done Sent false
35360 10244 Acked Read 3 Done Done Sent false
35361 10245 Acked Read 4 Done Done Sent false
35362 10246 Acked Flush 0 Done Done Sent false
35363 10247 Acked Read 8 Done Done Sent false
35364 10248 Acked Read 5 Done Done Sent false
35365 10249 Acked Read 2 Done Done Sent false
35366 10250 Acked Read 8 Done Done Sent false
35367 10251 Acked Read 7 Done Done Sent false
35368 10252 Acked Flush 0 Done Done Sent false
35369 10253 Acked Write 1 Done Done Sent false
35370 10254 Acked Write 10 Done Done Sent false
35371 10255 Acked Read 9 Done Done Sent false
35372 10256 Acked Read 10 Done Done Sent false
35373 10257 Acked Read 2 Done Done Sent false
35374 10258 Acked Write 4 Done Done Sent false
35375 10259 Acked Write 10 Done Done Sent false
35376 10260 Acked Read 10 Done Done Sent false
35377 10261 Acked Write 9 Done Done Sent false
35378 10262 Acked Read 1 Done Done Sent false
35379 10263 Acked Read 9 Done Done Sent false
35380 10264 Acked Write 9 Done Done Sent false
35381 10265 Acked Write 9 Done Done Sent false
35382 10266 Acked Write 10 Done Done Sent false
35383 10267 Acked Write 8 Done Done Sent false
35384 10268 Acked Read 4 Done Done Sent false
35385 10269 Acked Read 2 Done Done Sent false
35386 10270 Acked Read 9 Done Done Sent false
35387 10271 Acked Write 9 Done Done Sent false
35388 10272 Acked Read 10 Done Done Sent false
35389 10273 Acked Write 10 Done Done Sent false
35390 10274 Acked Read 9 Done Done Sent false
35391 10275 Acked Write 6 Done Done Sent false
35392 10276 Acked Write 6 Done Done Sent false
35393 10277 Acked Write 3 Done Done Sent false
35394 10278 Acked Read 6 Done Done Sent false
35395 10279 Acked Write 5 Done Done Sent false
35396 10280 Acked Read 6 Done Done Sent false
35397 10281 Acked Read 6 Done Done Sent false
35398 10282 Acked Write 1 Done Done Sent false
35399 10283 Acked Write 8 Done Done Sent false
35400 10284 Acked Read 5 Done Done Sent false
35401 10285 Acked Write 5 Done Done Sent false
35402 10286 Acked Read 5 Done Done Sent false
35403 10287 Acked Write 5 Done Done Sent false
35404 10288 Acked Write 8 Done Done Sent false
35405 10289 Acked Write 10 Done Done Sent false
35406 10290 Acked Write 6 Done Done Sent false
35407 10291 Acked Flush 0 Done Done Sent false
35408 10292 Acked Read 5 Done Done Sent false
35409 10293 Acked Read 8 Done Done Sent false
35410 10294 Acked Read 8 Done Done Sent false
35411 10295 Acked Read 1 Done Done Sent false
35412 10296 Acked Write 9 Done Done Sent false
35413 10297 Acked Write 6 Done Done Sent false
35414 10298 Acked Write 9 Done Done Sent false
35415 10299 Acked Write 6 Done Done Sent false
35416 10300 Acked Read 10 Done Done Sent false
35417 10301 Acked Read 8 Done Done Sent false
35418 10302 Acked Read 4 Done Done Sent false
35419 10303 Acked Read 8 Done Done Sent false
35420 10304 Acked Read 3 Done Done Sent false
35421 10305 Acked Write 2 Done Done Sent false
35422 10306 Acked Flush 0 Done Done Sent false
35423 10307 Acked Read 3 Done Done Sent false
35424 10308 Acked Read 1 Done Done Sent false
35425 10309 Acked Write 1 Done Done Sent false
35426 10310 Acked Write 3 Done Done Sent false
35427 10311 Acked Write 5 Done Done Sent false
35428 10312 Acked Write 4 Done Done Sent false
35429 10313 Acked Read 6 Done Done Sent false
35430 10314 Acked Write 2 Done Done Sent false
35431 10315 Acked Write 5 Done Done Sent false
35432 10316 Acked Read 1 Done Done Sent false
35433 10317 Acked Read 8 Done Done Sent false
35434 10318 Acked Write 2 Done Done Sent false
35435 10319 Acked Write 2 Done Done Sent false
35436 10320 Acked Read 8 Done Done Sent false
35437 10321 Acked Write 6 Done Done Sent false
35438 10322 Acked Read 9 Done Done Sent false
35439 10323 Acked Read 8 Done Done Sent false
35440 10324 Acked Read 4 Done Done Sent false
35441 10325 Acked Write 3 Done Done Sent false
35442 10326 Acked Write 1 Done Done Sent false
35443 10327 Acked Read 1 Done Done Sent false
35444 10328 Acked Write 5 Done Done Sent false
35445 10329 Acked Read 1 Done Done Sent false
35446 10330 Acked Read 4 Done Done Sent false
35447 10331 Acked Flush 0 Done Done Sent false
35448 10332 Acked Read 7 Done Done Sent false
35449 10333 Acked Read 9 Done Done Sent false
35450 10334 Acked Read 2 Done Done Sent false
35451 10335 Acked Write 9 Done Done Sent false
35452 10336 Acked Flush 0 Done Done Sent false
35453 10337 Acked Write 1 Done Done Sent false
35454 10338 Acked Read 7 Done Done Sent false
35455 10339 Acked Read 7 Done Done Sent false
35456 10340 Acked Read 9 Done Done Sent false
35457 10341 Acked Read 1 Done Done Sent false
35458 10342 Acked Write 6 Done Done Sent false
35459 10343 Acked Write 4 Done Done Sent false
35460 10344 Acked Read 2 Done Done Sent false
35461 10345 Acked Write 2 Done Done Sent false
35462 10346 Acked Read 9 Done Done Sent false
35463 10347 Acked Flush 0 Done Done Sent false
35464 10348 Acked Read 4 Done Done Sent false
35465 10349 Acked Read 7 Done Done Sent false
35466 10350 Acked Read 10 Done Done Sent false
35467 10351 Acked Read 3 Done Done Sent false
35468 10352 Acked Write 1 Done Done Sent false
35469 10353 Acked Read 8 Done Done Sent false
35470 10354 Acked Read 2 Done Done Sent false
35471 10355 Acked Read 6 Done Done Sent false
35472 10356 Acked Read 5 Done Done Sent false
35473 10357 Acked Read 4 Done Done Sent false
35474 10358 Acked Write 8 Done Done Sent false
35475 10359 Acked Write 10 Done Done Sent false
35476 10360 Acked Write 4 Done Done Sent false
35477 10361 Acked Read 4 Done Done Sent false
35478 10362 Acked Read 8 Done Done Sent false
35479 10363 Acked Write 8 Done Done Sent false
35480 10364 Acked Read 6 Done Done Sent false
35481 10365 Acked Write 5 Done Done Sent false
35482 10366 Acked Write 9 Done Done Sent false
35483 10367 Acked Write 2 Done Done Sent false
35484 10368 Acked Write 5 Done Done Sent false
35485 10369 Acked Flush 0 Done Done Sent false
35486 10370 Acked Write 8 Done Done Sent false
35487 10371 Acked Read 8 Done Done Sent false
35488 10372 Acked Write 8 Done Done Sent false
35489 10373 Acked Flush 0 Done Done Sent false
35490 STATES DS:0 DS:1 DS:2 TOTAL
35491 Sent 0 0 302 302
35492 Done 302 302 0 604
35493 Skipped 0 0 0 0
35494 Error 0 0 0 0
35495 Last Flush: 10373 10373 10071
35496 Downstairs last five completed: 10071 10070 10069 10068 10067
35497 Upstairs last five completed: 10373 10372 10371 10370 10369
35498 CLIENT: Up:0 ds:302 act:2
354992024-11-07T14:10:44.699ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
355002024-11-07T14:10:44.699ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63760) = io task client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
355012024-11-07T14:10:44.700ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
355022024-11-07T14:10:44.701ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
355032024-11-07T14:10:44.701ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
355042024-11-07T14:10:44.701ZINFOcrucible: send last flush ID to this DS: 10071 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
355052024-11-07T14:10:44.701ZINFOcrucible: Replied this last flush ID: 10071 = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
355062024-11-07T14:10:44.701ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
355072024-11-07T14:10:44.701ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
355082024-11-07T14:10:44.701ZINFOcrucible: [2] client re-new 302 jobs since flush 10071 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
355092024-11-07T14:10:44.701ZINFOcrucible: [2] Replayed 302 jobs since flush: 10071 = downstairs session_id = ac74b8f4-f0f2-4273-8fc2-10240de40f73
35510 ----------------------------------------------------------------
35511 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
35512 Downstairs last five completed: 10373 10372 10371 10370 10369
35513 Upstairs last five completed: 10373 10372 10371 10370 10369
35514 CLIENT: Up:0 ds:0 act:3
35515 Replay: All jobs finished
35516 Test replay has completed
35517 Wrote out file "/tmp/test_replay_verify.log"
35518 CLIENT: Tests done. All submitted work has been ACK'd
35519 ----------------------------------------------------------------
35520 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
35521 Downstairs last five completed: 10373 10372 10371 10370 10369
35522 Upstairs last five completed: 10373 10372 10371 10370 10369
35523 CLIENT: Up:0 ds:0 act:3
35524 CLIENT: All jobs finished, all DS active.
35525 Replays:30 time: 7:25
35526 Do final verify
35527 Nov 07 14:10:47.061 INFO accepted connection, remote_addr: 127.0.0.1:60460, local_addr: 127.0.0.1:9998
35528 Nov 07 14:10:47.062 INFO request completed, latency_us: 239, response_code: 200, uri: /regioninfo, method: GET, req_id: 4e3a4bd9-5537-417d-aad8-e348316f0d9c, remote_addr: 127.0.0.1:60460, local_addr: 127.0.0.1:9998
355292024-11-07T14:10:47.063ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
35530 Nov 07 14:10:47.063 INFO request completed, latency_us: 95, response_code: 200, uri: /regioncount, method: GET, req_id: d7ef0e8b-7c91-4ad0-bb66-e069423a74ce, remote_addr: 127.0.0.1:60460, local_addr: 127.0.0.1:9998
355312024-11-07T14:10:47.063ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
35532 Nov 07 14:10:47.064 INFO request completed, latency_us: 115, response_code: 200, uri: /port/cid/0, method: GET, req_id: b9d27cdc-4557-446b-b8a5-b6177257f5c5, remote_addr: 127.0.0.1:60460, local_addr: 127.0.0.1:9998
35533 Nov 07 14:10:47.064 INFO request completed, latency_us: 95, response_code: 200, uri: /port/cid/1, method: GET, req_id: af83a991-53b8-47d2-bd91-015f2ee65e33, remote_addr: 127.0.0.1:60460, local_addr: 127.0.0.1:9998
35534 Nov 07 14:10:47.065 INFO request completed, latency_us: 97, response_code: 200, uri: /port/cid/2, method: GET, req_id: f70b3f79-50bd-4b3f-b2be-181ee2bb026f, remote_addr: 127.0.0.1:60460, local_addr: 127.0.0.1:9998
355352024-11-07T14:10:47.065ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
355362024-11-07T14:10:47.065ZINFOcrucible: Upstairs starts
355372024-11-07T14:10:47.065ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "311c3ca2c97d568407ae06780269d96fcfdd3a7c", git_commit_timestamp: "2024-11-07T13:47:04.000000000Z", git_branch: "renovate/tokio-1.x-lockfile", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
355382024-11-07T14:10:47.065ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
355392024-11-07T14:10:47.065ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
355402024-11-07T14:10:47.066ZINFOcrucible: Crucible a40ff5ec-dc5b-4253-b98e-25a4316c2ba6 has session id: 8019daaa-36af-4425-aa64-abfbc1fe8a98 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355412024-11-07T14:10:47.066ZINFOcrucible: Upstairs opts: Upstairs UUID: a40ff5ec-dc5b-4253-b98e-25a4316c2ba6, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355422024-11-07T14:10:47.066ZINFOcrucible: Crucible stats registered with UUID: a40ff5ec-dc5b-4253-b98e-25a4316c2ba6 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355432024-11-07T14:10:47.100ZINFOcrucible: Downstairs in volume = 3 task = crutest
355442024-11-07T14:10:47.100ZINFOcrucible: The guest has requested activation with gen:3
355452024-11-07T14:10:47.100ZINFOcrucible: a40ff5ec-dc5b-4253-b98e-25a4316c2ba6 active request set session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355462024-11-07T14:10:47.100ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355472024-11-07T14:10:47.100ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355482024-11-07T14:10:47.100ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355492024-11-07T14:10:47.100ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355502024-11-07T14:10:47.100ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355512024-11-07T14:10:47.100ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355522024-11-07T14:10:47.100ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355532024-11-07T14:10:47.100ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355542024-11-07T14:10:47.100ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355552024-11-07T14:10:47.100ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51670) = io task client = 1 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355562024-11-07T14:10:47.100ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37624) = io task client = 2 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355572024-11-07T14:10:47.100ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40466) = io task client = 0 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355582024-11-07T14:10:47.101ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355592024-11-07T14:10:47.101ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355602024-11-07T14:10:47.101ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355612024-11-07T14:10:47.101ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355622024-11-07T14:10:47.101ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355632024-11-07T14:10:47.101ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355642024-11-07T14:10:47.102ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355652024-11-07T14:10:47.102ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355662024-11-07T14:10:47.102ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355672024-11-07T14:10:47.102ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355682024-11-07T14:10:47.102ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355692024-11-07T14:10:47.102ZINFOcrucible: Waiting for more clients to be ready session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355702024-11-07T14:10:47.103ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355712024-11-07T14:10:47.103ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355722024-11-07T14:10:47.103ZINFOcrucible: Waiting for more clients to be ready session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355732024-11-07T14:10:47.103ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355742024-11-07T14:10:47.103ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355752024-11-07T14:10:47.103ZINFOcrucible: [0]R flush_numbers[0..12]: [970, 991, 970, 995, 990, 978, 979, 990, 970, 990, 993, 956] = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355762024-11-07T14:10:47.103ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355772024-11-07T14:10:47.103ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355782024-11-07T14:10:47.103ZINFOcrucible: [1]R flush_numbers[0..12]: [970, 991, 970, 995, 990, 978, 979, 990, 970, 990, 993, 956] = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355792024-11-07T14:10:47.103ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355802024-11-07T14:10:47.103ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355812024-11-07T14:10:47.103ZINFOcrucible: [2]R flush_numbers[0..12]: [970, 991, 970, 995, 990, 978, 979, 990, 970, 990, 993, 956] = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355822024-11-07T14:10:47.103ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355832024-11-07T14:10:47.103ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355842024-11-07T14:10:47.103ZINFOcrucible: Max found gen is 3 = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355852024-11-07T14:10:47.103ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355862024-11-07T14:10:47.103ZINFOcrucible: Next flush: 997 = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355872024-11-07T14:10:47.103ZINFOcrucible: All extents match = downstairs session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355882024-11-07T14:10:47.103ZINFOcrucible: No downstairs reconciliation required session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355892024-11-07T14:10:47.103ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
35590 {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible"Wait for a query_work_queue command to finish before sending IO
35591 ,"level":30,"time":"2024-11-07T14:10:47.103385731Z","hostname":"w-01JC3EWTX1BFCJ6B7SGGW4W0MF","pid":1235,"client":"1","":"downstairs","session_id":"8019daaa-36af-4425-aa64-abfbc1fe8a98"}
355922024-11-07T14:10:47.103ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355932024-11-07T14:10:47.103ZINFOcrucible: All required reconciliation work is completed session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355942024-11-07T14:10:47.103ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355952024-11-07T14:10:47.103ZINFOcrucible: a40ff5ec-dc5b-4253-b98e-25a4316c2ba6 is now active with session: 8019daaa-36af-4425-aa64-abfbc1fe8a98 session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355962024-11-07T14:10:47.103ZINFOcrucible: Set Active after no reconciliation session_id = 8019daaa-36af-4425-aa64-abfbc1fe8a98
355972024-11-07T14:10:47.103ZINFOcrucible: The guest has finished waiting for activation with:3
35598 Region: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
35599 Loading write count information from file "/tmp/test_replay_verify.log"
35600 Read and Verify all blocks (0..5000 range:false)
35601 Wrote out file "/tmp/test_replay_verify.log"
35602 Verify test completed
35603 Wrote out file "/tmp/test_replay_verify.log"
35604 CLIENT: Tests done. All submitted work has been ACK'd
35605 ----------------------------------------------------------------
35606 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
35607 States: Active Active Active
35608 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35609 1000 Acked Read 256 Done Done Done false
35610 1001 Acked Read 256 Done Done Done false
35611 1002 Acked Read 256 Done Done Done false
35612 1003 Acked Read 256 Done Done Done false
35613 1004 Acked Read 256 Done Done Done false
35614 1005 Acked Read 256 Done Done Done false
35615 1006 Acked Read 256 Done Done Done false
35616 1007 Acked Read 256 Done Done Done false
35617 1008 Acked Read 256 Done Done Done false
35618 1009 Acked Read 256 Done Done Done false
35619 1010 Acked Read 256 Done Done Done false
35620 1011 Acked Read 256 Done Done Done false
35621 1012 Acked Read 256 Done Done Done false
35622 1013 Acked Read 256 Done Done Done false
35623 1014 Acked Read 256 Done Done Done false
35624 1015 Acked Read 256 Done Done Done false
35625 1016 Acked Read 256 Done Done Done false
35626 1017 Acked Read 256 Done Done Done false
35627 1018 Acked Read 136 Done Done Done false
35628 1019 Acked Read 256 Done Done Done false
35629 STATES DS:0 DS:1 DS:2 TOTAL
35630 Sent 0 0 0 0
35631 Done 20 20 20 60
35632 Skipped 0 0 0 0
35633 Error 0 0 0 0
35634 Last Flush: 0 0 0
35635 Downstairs last five completed:
35636 Upstairs last five completed: 1019 1018 1017 1016 1015
35637 CLIENT: Up:0 ds:20 act:3
35638 ----------------------------------------------------------------
35639 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
35640 Downstairs last five completed: 1020 1019 1018 1017 1016
35641 Upstairs last five completed: 1020 1019 1018 1017 1016
35642 CLIENT: Up:0 ds:0 act:3
35643 CLIENT: All crucible jobs finished, exiting program
356442024-11-07T14:10:51.511ZWARNcrucible: Guest handle has been dropped
35645 Nov 07 14:10:51.576 INFO accepted connection, remote_addr: 127.0.0.1:36043, local_addr: 127.0.0.1:9998
35646 Main task has work to do, go find it
35647 got dsc Shutdown
35648 Shutdown
35649 Shut it down
35650 Nov 07 14:10:51.577 INFO request completed, latency_us: 199, response_code: 204, uri: /shutdown, method: POST, req_id: dee1462f-cfd4-4a56-bc5c-41892533d81f, remote_addr:[8810] Disable keep_running
35651 [8810] Got stop action so:false kr:false
35652 127.0.0.1:36043, local_addr: 127.0.0.1:9998
35653 [8820] Disable keep_running
35654 [8820] Got stop action so:false kr:false
35655 [8830] Disable keep_running
35656 [8830] Got stop action so:false kr:false
35657 [8810] Exited with: ExitStatus(unix_wait_status(9))
35658 [8830] Exited with: ExitStatus(unix_wait_status(9))
35659 [8810][0] reports Exit
35660 [8830][2] reports Exit
35661 [8820] Exited with: ExitStatus(unix_wait_status(9))
35662 [8820][1] reports Exit
35663 November 7, 2024 at 02:10:56 PM UTC Test ends with 0