Max level shown:
12026-07-30T03:05:12.731ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::a]:60582
22026-07-30T03:05:12.731ZINFOcrucible-agent (datafile): region d030f23e-be36-4d3e-9f97-1bfe10e65ed0 state: Requested
32026-07-30T03:05:12.732ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-07-30T03:05:12.732ZINFOcrucible-agent (dropshot): request completed latency_us = 1109 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60582 req_id = b0f1ce24-c3ab-443e-af11-0b80ba27ec96 response_code = 200 uri = /crucible/0/regions
52026-07-30T03:05:12.741ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/d030f23e-be36-4d3e-9f97-1bfe10e65ed0
62026-07-30T03:05:12.741ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/d030f23e-be36-4d3e-9f97-1bfe10e65ed0
72026-07-30T03:05:12.793ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d030f23e-be36-4d3e-9f97-1bfe10e65ed0"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d030f23e-be36-4d3e-9f97-1bfe10e65ed0" region = d030f23e-be36-4d3e-9f97-1bfe10e65ed0
82026-07-30T03:05:12.793ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d030f23e-be36-4d3e-9f97-1bfe10e65ed0" "--data" "/data/regions/d030f23e-be36-4d3e-9f97-1bfe10e65ed0" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = d030f23e-be36-4d3e-9f97-1bfe10e65ed0
92026-07-30T03:05:12.868ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60582 req_id = f6a8d08c-3300-4c12-8b99-ff2371f0de21 response_code = 200 uri = /crucible/0/regions
102026-07-30T03:05:12.868ZINFOcrucible-agent (worker): region files created ok region = d030f23e-be36-4d3e-9f97-1bfe10e65ed0
112026-07-30T03:05:12.868ZINFOcrucible-agent (datafile): region d030f23e-be36-4d3e-9f97-1bfe10e65ed0 state: Requested -> Created
122026-07-30T03:05:12.868ZINFOcrucible-agent (worker): applying SMF actions post create...
132026-07-30T03:05:12.875ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d030f23e-be36-4d3e-9f97-1bfe10e65ed0
142026-07-30T03:05:12.882ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d030f23e-be36-4d3e-9f97-1bfe10e65ed0
152026-07-30T03:05:12.892ZINFOcrucible-agent (worker): creating config property group
162026-07-30T03:05:12.895ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d030f23e-be36-4d3e-9f97-1bfe10e65ed0
172026-07-30T03:05:12.895ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d030f23e-be36-4d3e-9f97-1bfe10e65ed0
182026-07-30T03:05:12.895ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192026-07-30T03:05:12.895ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
202026-07-30T03:05:12.895ZINFOcrucible-agent (worker): commit
212026-07-30T03:05:12.899ZINFOcrucible-agent (worker): ok!
222026-07-30T03:05:12.904ZINFOcrucible-agent (worker): SMF ok!
232026-07-30T03:05:13.160ZINFOcrucible-agent (dropshot): request completed latency_us = 58 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60582 req_id = acf4464a-0642-4f33-9959-f4383c842c77 response_code = 200 uri = /crucible/0/regions
242026-07-30T03:05:14.637ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60582 req_id = 8df8a1ab-61ca-4908-805d-7b67a46e9263 response_code = 200 uri = /crucible/0/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0
252026-07-30T03:05:14.640ZINFOcrucible-agent (datafile): path is "/data/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0"
262026-07-30T03:05:14.648ZINFOcrucible-agent (datafile): dataset is oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0
272026-07-30T03:05:14.655ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0@10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5
282026-07-30T03:05:14.655ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0", "10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5"]
292026-07-30T03:05:14.667ZINFOcrucible-agent (dropshot): request completed latency_us = 26995 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60582 req_id = 3273a8fe-c641-4a24-8c64-ba7d4644416c response_code = 200 uri = /crucible/0/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0/snapshots/10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5
302026-07-30T03:05:14.672ZINFOcrucible-agent (datafile): path is "/data/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0"
312026-07-30T03:05:14.678ZINFOcrucible-agent (datafile): dataset is oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0
322026-07-30T03:05:14.685ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0@10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5
332026-07-30T03:05:14.685ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0", "10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5"]
342026-07-30T03:05:14.692ZINFOcrucible-agent (datafile): requesting running snapshot 662360bc-512c-4eef-8c7f-81378a2fa1d0-10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5 state: Requested
352026-07-30T03:05:14.695ZINFOcrucible-agent (worker): applying SMF actions for region 662360bc-512c-4eef-8c7f-81378a2fa1d0 running snapshot 10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5 (state Requested)...
362026-07-30T03:05:14.695ZINFOcrucible-agent (dropshot): request completed latency_us = 23217 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60582 req_id = 16768576-3529-4474-9770-087fa51e2a5e response_code = 200 uri = /crucible/0/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0/snapshots/10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5/run
372026-07-30T03:05:14.700ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-662360bc-512c-4eef-8c7f-81378a2fa1d0-10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5
382026-07-30T03:05:14.704ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-662360bc-512c-4eef-8c7f-81378a2fa1d0-10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5
392026-07-30T03:05:14.715ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-662360bc-512c-4eef-8c7f-81378a2fa1d0-10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5
402026-07-30T03:05:14.715ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0/.zfs/snapshot/10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5
412026-07-30T03:05:14.715ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
422026-07-30T03:05:14.715ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
432026-07-30T03:05:14.715ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
442026-07-30T03:05:14.715ZINFOcrucible-agent (worker): commit
452026-07-30T03:05:14.719ZINFOcrucible-agent (worker): ok!
462026-07-30T03:05:14.723ZINFOcrucible-agent (worker): SMF ok!
472026-07-30T03:05:14.723ZINFOcrucible-agent (datafile): region 662360bc-512c-4eef-8c7f-81378a2fa1d0 running snapshot 10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5 state: Requested -> Created
482026-07-30T03:05:14.966ZINFOcrucible-agent (datafile): path is "/data/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0"
492026-07-30T03:05:14.976ZINFOcrucible-agent (datafile): dataset is oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0
502026-07-30T03:05:14.983ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0@10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5
512026-07-30T03:05:14.983ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f68bf5b8-7eec-44b3-966b-5537b65273db/crucible/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0", "10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5"]
522026-07-30T03:05:14.990ZINFOcrucible-agent (dropshot): request completed latency_us = 24098 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60582 req_id = 9e913910-c4cc-448b-bab7-0a508ac7c741 response_code = 200 uri = /crucible/0/regions/662360bc-512c-4eef-8c7f-81378a2fa1d0/snapshots/10d4c3dc-e49a-405f-b5d5-8f1f519f2cc5/run