Max level shown:
12026-11-05T02:19:49.533ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::b]:47337
22026-11-05T02:19:49.533ZINFOcrucible-agent (datafile): region 61835337-b480-4d99-b073-4cc3fb69bf40 state: Requested
32026-11-05T02:19:49.534ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-11-05T02:19:49.534ZINFOcrucible-agent (dropshot): request completed latency_us = 635 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:47337 req_id = 47020683-aa1f-47dd-ab7c-8f53bc8071f9 response_code = 200 uri = /crucible/0/regions
52026-11-05T02:19:49.546ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/61835337-b480-4d99-b073-4cc3fb69bf40
62026-11-05T02:19:49.546ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/61835337-b480-4d99-b073-4cc3fb69bf40
72026-11-05T02:19:49.587ZINFOcrucible-agent (worker): creating region Region { id: RegionId("61835337-b480-4d99-b073-4cc3fb69bf40"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/61835337-b480-4d99-b073-4cc3fb69bf40" region = 61835337-b480-4d99-b073-4cc3fb69bf40
82026-11-05T02:19:49.587ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "61835337-b480-4d99-b073-4cc3fb69bf40" "--data" "/data/regions/61835337-b480-4d99-b073-4cc3fb69bf40" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 61835337-b480-4d99-b073-4cc3fb69bf40
92026-11-05T02:19:49.640ZINFOcrucible-agent (worker): region files created ok region = 61835337-b480-4d99-b073-4cc3fb69bf40
102026-11-05T02:19:49.640ZINFOcrucible-agent (datafile): region 61835337-b480-4d99-b073-4cc3fb69bf40 state: Requested -> Created
112026-11-05T02:19:49.641ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-11-05T02:19:49.641ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-61835337-b480-4d99-b073-4cc3fb69bf40
132026-11-05T02:19:49.656ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-61835337-b480-4d99-b073-4cc3fb69bf40
142026-11-05T02:19:49.661ZINFOcrucible-agent (worker): creating config property group
152026-11-05T02:19:49.663ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-61835337-b480-4d99-b073-4cc3fb69bf40
162026-11-05T02:19:49.663ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/61835337-b480-4d99-b073-4cc3fb69bf40
172026-11-05T02:19:49.663ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-11-05T02:19:49.663ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
192026-11-05T02:19:49.663ZINFOcrucible-agent (worker): commit
202026-11-05T02:19:49.665ZINFOcrucible-agent (worker): ok!
212026-11-05T02:19:49.669ZINFOcrucible-agent (worker): SMF ok!
222026-11-05T02:19:49.757ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:47337 req_id = 3b549412-0a51-41c3-aa4e-eb2bd0d38b21 response_code = 200 uri = /crucible/0/regions
232026-11-05T02:19:50.944ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:47337 req_id = e7e17364-1563-4c2f-8b00-3875204a2c22 response_code = 200 uri = /crucible/0/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94
242026-11-05T02:19:50.950ZINFOcrucible-agent (datafile): path is "/data/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94"
252026-11-05T02:19:50.956ZINFOcrucible-agent (datafile): dataset is oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94
262026-11-05T02:19:50.962ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94@fdde8712-f81f-427a-9439-429678eb2f11
272026-11-05T02:19:50.962ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94", "fdde8712-f81f-427a-9439-429678eb2f11"]
282026-11-05T02:19:50.973ZINFOcrucible-agent (dropshot): request completed latency_us = 25935 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:47337 req_id = bb6176e1-e900-44d9-9f7b-90e42a8c3488 response_code = 200 uri = /crucible/0/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94/snapshots/fdde8712-f81f-427a-9439-429678eb2f11
292026-11-05T02:19:50.977ZINFOcrucible-agent (datafile): path is "/data/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94"
302026-11-05T02:19:50.982ZINFOcrucible-agent (datafile): dataset is oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94
312026-11-05T02:19:50.988ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94@fdde8712-f81f-427a-9439-429678eb2f11
322026-11-05T02:19:50.988ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94", "fdde8712-f81f-427a-9439-429678eb2f11"]
332026-11-05T02:19:50.994ZINFOcrucible-agent (datafile): requesting running snapshot 816fe0c5-6743-451e-88dd-56caf6f8dd94-fdde8712-f81f-427a-9439-429678eb2f11 state: Requested
342026-11-05T02:19:50.994ZINFOcrucible-agent (worker): applying SMF actions for region 816fe0c5-6743-451e-88dd-56caf6f8dd94 running snapshot fdde8712-f81f-427a-9439-429678eb2f11 (state Requested)...
352026-11-05T02:19:50.994ZINFOcrucible-agent (dropshot): request completed latency_us = 17644 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:47337 req_id = addfafc6-e8b5-4f9f-af32-b994c8ec90a8 response_code = 200 uri = /crucible/0/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94/snapshots/fdde8712-f81f-427a-9439-429678eb2f11/run
362026-11-05T02:19:50.997ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-816fe0c5-6743-451e-88dd-56caf6f8dd94-fdde8712-f81f-427a-9439-429678eb2f11
372026-11-05T02:19:50.998ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-816fe0c5-6743-451e-88dd-56caf6f8dd94-fdde8712-f81f-427a-9439-429678eb2f11
382026-11-05T02:19:51.003ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-816fe0c5-6743-451e-88dd-56caf6f8dd94-fdde8712-f81f-427a-9439-429678eb2f11
392026-11-05T02:19:51.004ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94/.zfs/snapshot/fdde8712-f81f-427a-9439-429678eb2f11
402026-11-05T02:19:51.004ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-11-05T02:19:51.004ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-11-05T02:19:51.004ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
432026-11-05T02:19:51.004ZINFOcrucible-agent (worker): commit
442026-11-05T02:19:51.006ZINFOcrucible-agent (worker): ok!
452026-11-05T02:19:51.009ZINFOcrucible-agent (worker): SMF ok!
462026-11-05T02:19:51.009ZINFOcrucible-agent (datafile): region 816fe0c5-6743-451e-88dd-56caf6f8dd94 running snapshot fdde8712-f81f-427a-9439-429678eb2f11 state: Requested -> Created
472026-11-05T02:19:51.204ZINFOcrucible-agent (datafile): path is "/data/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94"
482026-11-05T02:19:51.211ZINFOcrucible-agent (datafile): dataset is oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94
492026-11-05T02:19:51.217ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94@fdde8712-f81f-427a-9439-429678eb2f11
502026-11-05T02:19:51.217ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ff2de8c9-5f71-492d-a6d3-a4277f49918b/crucible/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94", "fdde8712-f81f-427a-9439-429678eb2f11"]
512026-11-05T02:19:51.222ZINFOcrucible-agent (dropshot): request completed latency_us = 17892 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:47337 req_id = 641c3ee8-a867-446d-9f4c-e056d4ccb0fd response_code = 200 uri = /crucible/0/regions/816fe0c5-6743-451e-88dd-56caf6f8dd94/snapshots/fdde8712-f81f-427a-9439-429678eb2f11/run