Max level shown:
12026-04-10T22:34:30.508ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:58229
22026-04-10T22:34:30.508ZINFOcrucible-agent (datafile): region b1313b88-e7c2-4bb1-bd2b-e050baed7102 state: Requested
32026-04-10T22:34:30.508ZINFOcrucible-agent (dropshot): request completed latency_us = 621 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58229 req_id = c2924b95-9598-4822-ae8e-179b2e900ca8 response_code = 200 uri = /crucible/0/regions
42026-04-10T22:34:30.508ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-04-10T22:34:30.517ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/b1313b88-e7c2-4bb1-bd2b-e050baed7102
62026-04-10T22:34:30.517ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/b1313b88-e7c2-4bb1-bd2b-e050baed7102
72026-04-10T22:34:30.571ZINFOcrucible-agent (worker): creating region Region { id: RegionId("b1313b88-e7c2-4bb1-bd2b-e050baed7102"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/b1313b88-e7c2-4bb1-bd2b-e050baed7102" region = b1313b88-e7c2-4bb1-bd2b-e050baed7102
82026-04-10T22:34:30.571ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "b1313b88-e7c2-4bb1-bd2b-e050baed7102" "--data" "/data/regions/b1313b88-e7c2-4bb1-bd2b-e050baed7102" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = b1313b88-e7c2-4bb1-bd2b-e050baed7102
92026-04-10T22:34:30.659ZINFOcrucible-agent (worker): region files created ok region = b1313b88-e7c2-4bb1-bd2b-e050baed7102
102026-04-10T22:34:30.659ZINFOcrucible-agent (datafile): region b1313b88-e7c2-4bb1-bd2b-e050baed7102 state: Requested -> Created
112026-04-10T22:34:30.659ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-04-10T22:34:30.680ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-b1313b88-e7c2-4bb1-bd2b-e050baed7102
132026-04-10T22:34:30.691ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-b1313b88-e7c2-4bb1-bd2b-e050baed7102
142026-04-10T22:34:30.704ZINFOcrucible-agent (worker): creating config property group
152026-04-10T22:34:30.707ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-b1313b88-e7c2-4bb1-bd2b-e050baed7102
162026-04-10T22:34:30.707ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b1313b88-e7c2-4bb1-bd2b-e050baed7102
172026-04-10T22:34:30.707ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-04-10T22:34:30.707ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
192026-04-10T22:34:30.707ZINFOcrucible-agent (worker): commit
202026-04-10T22:34:30.709ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58229 req_id = 03b8663c-193d-4349-924d-0f723fcbd41b response_code = 200 uri = /crucible/0/regions
212026-04-10T22:34:30.711ZINFOcrucible-agent (worker): ok!
222026-04-10T22:34:30.719ZINFOcrucible-agent (worker): SMF ok!
232026-04-10T22:34:32.991ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:58229 req_id = 95922795-a5fc-48f4-ad03-34e70b6c41a0 response_code = 200 uri = /crucible/0/regions/a52fb219-bb45-464f-a0dc-671b0e046e65
242026-04-10T22:34:32.995ZINFOcrucible-agent (datafile): path is "/data/regions/a52fb219-bb45-464f-a0dc-671b0e046e65"
252026-04-10T22:34:33.000ZINFOcrucible-agent (datafile): dataset is oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/a52fb219-bb45-464f-a0dc-671b0e046e65
262026-04-10T22:34:33.007ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/a52fb219-bb45-464f-a0dc-671b0e046e65@7c7854fe-198f-43b9-822d-69a32c45d0a6
272026-04-10T22:34:33.007ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/a52fb219-bb45-464f-a0dc-671b0e046e65", "7c7854fe-198f-43b9-822d-69a32c45d0a6"]
282026-04-10T22:34:33.015ZINFOcrucible-agent (dropshot): request completed latency_us = 20041 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:58229 req_id = a2bb8d6d-091c-41bf-ae5f-8c5a0df204bc response_code = 200 uri = /crucible/0/regions/a52fb219-bb45-464f-a0dc-671b0e046e65/snapshots/7c7854fe-198f-43b9-822d-69a32c45d0a6
292026-04-10T22:34:33.018ZINFOcrucible-agent (datafile): path is "/data/regions/a52fb219-bb45-464f-a0dc-671b0e046e65"
302026-04-10T22:34:33.024ZINFOcrucible-agent (datafile): dataset is oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/a52fb219-bb45-464f-a0dc-671b0e046e65
312026-04-10T22:34:33.030ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/a52fb219-bb45-464f-a0dc-671b0e046e65@7c7854fe-198f-43b9-822d-69a32c45d0a6
322026-04-10T22:34:33.030ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/a52fb219-bb45-464f-a0dc-671b0e046e65", "7c7854fe-198f-43b9-822d-69a32c45d0a6"]
332026-04-10T22:34:33.037ZINFOcrucible-agent (datafile): requesting running snapshot a52fb219-bb45-464f-a0dc-671b0e046e65-7c7854fe-198f-43b9-822d-69a32c45d0a6 state: Requested
342026-04-10T22:34:33.038ZINFOcrucible-agent (dropshot): request completed latency_us = 19373 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58229 req_id = 40410054-1d00-418d-8c17-82b375e46d0c response_code = 200 uri = /crucible/0/regions/a52fb219-bb45-464f-a0dc-671b0e046e65/snapshots/7c7854fe-198f-43b9-822d-69a32c45d0a6/run
352026-04-10T22:34:33.038ZINFOcrucible-agent (worker): applying SMF actions for region a52fb219-bb45-464f-a0dc-671b0e046e65 running snapshot 7c7854fe-198f-43b9-822d-69a32c45d0a6 (state Requested)...
362026-04-10T22:34:33.042ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-a52fb219-bb45-464f-a0dc-671b0e046e65-7c7854fe-198f-43b9-822d-69a32c45d0a6
372026-04-10T22:34:33.050ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-a52fb219-bb45-464f-a0dc-671b0e046e65-7c7854fe-198f-43b9-822d-69a32c45d0a6
382026-04-10T22:34:33.060ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-a52fb219-bb45-464f-a0dc-671b0e046e65-7c7854fe-198f-43b9-822d-69a32c45d0a6
392026-04-10T22:34:33.061ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a52fb219-bb45-464f-a0dc-671b0e046e65/.zfs/snapshot/7c7854fe-198f-43b9-822d-69a32c45d0a6
402026-04-10T22:34:33.061ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-04-10T22:34:33.061ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-04-10T22:34:33.061ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432026-04-10T22:34:33.061ZINFOcrucible-agent (worker): commit
442026-04-10T22:34:33.064ZINFOcrucible-agent (worker): ok!
452026-04-10T22:34:33.069ZINFOcrucible-agent (worker): SMF ok!
462026-04-10T22:34:33.069ZINFOcrucible-agent (datafile): region a52fb219-bb45-464f-a0dc-671b0e046e65 running snapshot 7c7854fe-198f-43b9-822d-69a32c45d0a6 state: Requested -> Created
472026-04-10T22:34:33.270ZINFOcrucible-agent (datafile): path is "/data/regions/a52fb219-bb45-464f-a0dc-671b0e046e65"
482026-04-10T22:34:33.277ZINFOcrucible-agent (datafile): dataset is oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/a52fb219-bb45-464f-a0dc-671b0e046e65
492026-04-10T22:34:33.283ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/a52fb219-bb45-464f-a0dc-671b0e046e65@7c7854fe-198f-43b9-822d-69a32c45d0a6
502026-04-10T22:34:33.283ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6af4d82f-2fb5-48a1-aaa0-87bd3bd414e6/crucible/regions/a52fb219-bb45-464f-a0dc-671b0e046e65", "7c7854fe-198f-43b9-822d-69a32c45d0a6"]
512026-04-10T22:34:33.289ZINFOcrucible-agent (dropshot): request completed latency_us = 18989 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58229 req_id = c91d7b5f-e470-414b-a8cd-9eb1536ce8cf response_code = 200 uri = /crucible/0/regions/a52fb219-bb45-464f-a0dc-671b0e046e65/snapshots/7c7854fe-198f-43b9-822d-69a32c45d0a6/run