Max level shown:
12026-04-17T23:40:01.953ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::a]:49854
22026-04-17T23:40:01.953ZINFOcrucible-agent (datafile): region 0527c57d-5b17-4285-a3be-94344fc01e66 state: Requested
32026-04-17T23:40:01.954ZINFOcrucible-agent (dropshot): request completed latency_us = 635 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49854 req_id = 103f88b6-6cf6-4613-a7c5-04983f1cd2ed response_code = 200 uri = /crucible/0/regions
42026-04-17T23:40:01.954ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-04-17T23:40:01.963ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/0527c57d-5b17-4285-a3be-94344fc01e66
62026-04-17T23:40:01.963ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/0527c57d-5b17-4285-a3be-94344fc01e66
72026-04-17T23:40:02.017ZINFOcrucible-agent (worker): creating region Region { id: RegionId("0527c57d-5b17-4285-a3be-94344fc01e66"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/0527c57d-5b17-4285-a3be-94344fc01e66" region = 0527c57d-5b17-4285-a3be-94344fc01e66
82026-04-17T23:40:02.017ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "0527c57d-5b17-4285-a3be-94344fc01e66" "--data" "/data/regions/0527c57d-5b17-4285-a3be-94344fc01e66" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 0527c57d-5b17-4285-a3be-94344fc01e66
92026-04-17T23:40:02.123ZINFOcrucible-agent (worker): region files created ok region = 0527c57d-5b17-4285-a3be-94344fc01e66
102026-04-17T23:40:02.123ZINFOcrucible-agent (datafile): region 0527c57d-5b17-4285-a3be-94344fc01e66 state: Requested -> Created
112026-04-17T23:40:02.123ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-04-17T23:40:02.123ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-0527c57d-5b17-4285-a3be-94344fc01e66
132026-04-17T23:40:02.133ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-0527c57d-5b17-4285-a3be-94344fc01e66
142026-04-17T23:40:02.157ZINFOcrucible-agent (worker): creating config property group
152026-04-17T23:40:02.161ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-0527c57d-5b17-4285-a3be-94344fc01e66
162026-04-17T23:40:02.161ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/0527c57d-5b17-4285-a3be-94344fc01e66
172026-04-17T23:40:02.161ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-04-17T23:40:02.161ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
192026-04-17T23:40:02.161ZINFOcrucible-agent (worker): commit
202026-04-17T23:40:02.165ZINFOcrucible-agent (worker): ok!
212026-04-17T23:40:02.181ZINFOcrucible-agent (worker): SMF ok!
222026-04-17T23:40:02.333ZINFOcrucible-agent (dropshot): request completed latency_us = 122 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49854 req_id = 44048b7d-4e77-41fc-a6b0-1ddb306f0237 response_code = 200 uri = /crucible/0/regions
232026-04-17T23:40:03.649ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49854 req_id = e93e553a-ac3e-4842-bc0a-c11cb60cc42f response_code = 200 uri = /crucible/0/regions/59159f1e-df59-4694-9e03-1ae385655f32
242026-04-17T23:40:03.652ZINFOcrucible-agent (datafile): path is "/data/regions/59159f1e-df59-4694-9e03-1ae385655f32"
252026-04-17T23:40:03.659ZINFOcrucible-agent (datafile): dataset is oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/59159f1e-df59-4694-9e03-1ae385655f32
262026-04-17T23:40:03.665ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/59159f1e-df59-4694-9e03-1ae385655f32@06a03600-e6e2-4f38-b7ad-4bddfff1ff57
272026-04-17T23:40:03.665ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/59159f1e-df59-4694-9e03-1ae385655f32", "06a03600-e6e2-4f38-b7ad-4bddfff1ff57"]
282026-04-17T23:40:03.674ZINFOcrucible-agent (dropshot): request completed latency_us = 21470 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:49854 req_id = 592d6bfd-2d5f-45fc-ab4c-d4610c237d36 response_code = 200 uri = /crucible/0/regions/59159f1e-df59-4694-9e03-1ae385655f32/snapshots/06a03600-e6e2-4f38-b7ad-4bddfff1ff57
292026-04-17T23:40:03.678ZINFOcrucible-agent (datafile): path is "/data/regions/59159f1e-df59-4694-9e03-1ae385655f32"
302026-04-17T23:40:03.684ZINFOcrucible-agent (datafile): dataset is oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/59159f1e-df59-4694-9e03-1ae385655f32
312026-04-17T23:40:03.690ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/59159f1e-df59-4694-9e03-1ae385655f32@06a03600-e6e2-4f38-b7ad-4bddfff1ff57
322026-04-17T23:40:03.690ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/59159f1e-df59-4694-9e03-1ae385655f32", "06a03600-e6e2-4f38-b7ad-4bddfff1ff57"]
332026-04-17T23:40:03.697ZINFOcrucible-agent (datafile): requesting running snapshot 59159f1e-df59-4694-9e03-1ae385655f32-06a03600-e6e2-4f38-b7ad-4bddfff1ff57 state: Requested
342026-04-17T23:40:03.697ZINFOcrucible-agent (dropshot): request completed latency_us = 19904 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49854 req_id = 564968f9-30a8-40b1-b29a-8cc5dd09b819 response_code = 200 uri = /crucible/0/regions/59159f1e-df59-4694-9e03-1ae385655f32/snapshots/06a03600-e6e2-4f38-b7ad-4bddfff1ff57/run
352026-04-17T23:40:03.697ZINFOcrucible-agent (worker): applying SMF actions for region 59159f1e-df59-4694-9e03-1ae385655f32 running snapshot 06a03600-e6e2-4f38-b7ad-4bddfff1ff57 (state Requested)...
362026-04-17T23:40:03.702ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-59159f1e-df59-4694-9e03-1ae385655f32-06a03600-e6e2-4f38-b7ad-4bddfff1ff57
372026-04-17T23:40:03.715ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-59159f1e-df59-4694-9e03-1ae385655f32-06a03600-e6e2-4f38-b7ad-4bddfff1ff57
382026-04-17T23:40:03.752ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-59159f1e-df59-4694-9e03-1ae385655f32-06a03600-e6e2-4f38-b7ad-4bddfff1ff57
392026-04-17T23:40:03.752ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/59159f1e-df59-4694-9e03-1ae385655f32/.zfs/snapshot/06a03600-e6e2-4f38-b7ad-4bddfff1ff57
402026-04-17T23:40:03.752ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-04-17T23:40:03.752ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-04-17T23:40:03.752ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
432026-04-17T23:40:03.752ZINFOcrucible-agent (worker): commit
442026-04-17T23:40:03.758ZINFOcrucible-agent (worker): ok!
452026-04-17T23:40:03.764ZINFOcrucible-agent (worker): SMF ok!
462026-04-17T23:40:03.764ZINFOcrucible-agent (datafile): region 59159f1e-df59-4694-9e03-1ae385655f32 running snapshot 06a03600-e6e2-4f38-b7ad-4bddfff1ff57 state: Requested -> Created
472026-04-17T23:40:03.963ZINFOcrucible-agent (datafile): path is "/data/regions/59159f1e-df59-4694-9e03-1ae385655f32"
482026-04-17T23:40:03.969ZINFOcrucible-agent (datafile): dataset is oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/59159f1e-df59-4694-9e03-1ae385655f32
492026-04-17T23:40:03.975ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/59159f1e-df59-4694-9e03-1ae385655f32@06a03600-e6e2-4f38-b7ad-4bddfff1ff57
502026-04-17T23:40:03.975ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_daec6a5f-ab5b-404b-b0b7-014c180583fc/crucible/regions/59159f1e-df59-4694-9e03-1ae385655f32", "06a03600-e6e2-4f38-b7ad-4bddfff1ff57"]
512026-04-17T23:40:03.981ZINFOcrucible-agent (dropshot): request completed latency_us = 18692 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:49854 req_id = ffe31188-f616-4f59-b9f0-9acabd404378 response_code = 200 uri = /crucible/0/regions/59159f1e-df59-4694-9e03-1ae385655f32/snapshots/06a03600-e6e2-4f38-b7ad-4bddfff1ff57/run