Max level shown:
12026-08-19T03:19:30.336ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::c]:42578
22026-08-19T03:19:30.336ZINFOcrucible-agent (datafile): region 52f9a5e3-24ff-4b77-a51d-13c059702f07 state: Requested
32026-08-19T03:19:30.337ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-08-19T03:19:30.337ZINFOcrucible-agent (dropshot): request completed latency_us = 893 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42578 req_id = c550baa5-4925-406f-921a-df69131fd8ba response_code = 200 uri = /crucible/0/regions
52026-08-19T03:19:30.347ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/52f9a5e3-24ff-4b77-a51d-13c059702f07
62026-08-19T03:19:30.347ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/52f9a5e3-24ff-4b77-a51d-13c059702f07
72026-08-19T03:19:30.394ZINFOcrucible-agent (worker): creating region Region { id: RegionId("52f9a5e3-24ff-4b77-a51d-13c059702f07"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/52f9a5e3-24ff-4b77-a51d-13c059702f07" region = 52f9a5e3-24ff-4b77-a51d-13c059702f07
82026-08-19T03:19:30.394ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "52f9a5e3-24ff-4b77-a51d-13c059702f07" "--data" "/data/regions/52f9a5e3-24ff-4b77-a51d-13c059702f07" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 52f9a5e3-24ff-4b77-a51d-13c059702f07
92026-08-19T03:19:30.446ZINFOcrucible-agent (worker): region files created ok region = 52f9a5e3-24ff-4b77-a51d-13c059702f07
102026-08-19T03:19:30.446ZINFOcrucible-agent (datafile): region 52f9a5e3-24ff-4b77-a51d-13c059702f07 state: Requested -> Created
112026-08-19T03:19:30.446ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-08-19T03:19:30.447ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-52f9a5e3-24ff-4b77-a51d-13c059702f07
132026-08-19T03:19:30.460ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-52f9a5e3-24ff-4b77-a51d-13c059702f07
142026-08-19T03:19:30.469ZINFOcrucible-agent (worker): creating config property group
152026-08-19T03:19:30.471ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-52f9a5e3-24ff-4b77-a51d-13c059702f07
162026-08-19T03:19:30.471ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/52f9a5e3-24ff-4b77-a51d-13c059702f07
172026-08-19T03:19:30.471ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-08-19T03:19:30.471ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
192026-08-19T03:19:30.471ZINFOcrucible-agent (worker): commit
202026-08-19T03:19:30.473ZINFOcrucible-agent (worker): ok!
212026-08-19T03:19:30.476ZINFOcrucible-agent (worker): SMF ok!
222026-08-19T03:19:30.699ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42578 req_id = 437a85cb-4c5f-412a-b079-f48126d646ce response_code = 200 uri = /crucible/0/regions
232026-08-19T03:19:31.920ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:42578 req_id = 7e45f94a-5f4d-4a49-b0d4-84ddcf667a72 response_code = 200 uri = /crucible/0/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5
242026-08-19T03:19:31.923ZINFOcrucible-agent (datafile): path is "/data/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5"
252026-08-19T03:19:31.929ZINFOcrucible-agent (datafile): dataset is oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5
262026-08-19T03:19:31.935ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5@3c128254-b43c-4098-ba8a-6c3866b6a582
272026-08-19T03:19:31.935ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5", "3c128254-b43c-4098-ba8a-6c3866b6a582"]
282026-08-19T03:19:31.945ZINFOcrucible-agent (dropshot): request completed latency_us = 22729 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:42578 req_id = c5568362-ca85-4fe7-b67b-531df5f64020 response_code = 200 uri = /crucible/0/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5/snapshots/3c128254-b43c-4098-ba8a-6c3866b6a582
292026-08-19T03:19:31.949ZINFOcrucible-agent (datafile): path is "/data/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5"
302026-08-19T03:19:31.955ZINFOcrucible-agent (datafile): dataset is oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5
312026-08-19T03:19:31.960ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5@3c128254-b43c-4098-ba8a-6c3866b6a582
322026-08-19T03:19:31.960ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5", "3c128254-b43c-4098-ba8a-6c3866b6a582"]
332026-08-19T03:19:31.967ZINFOcrucible-agent (datafile): requesting running snapshot 811f03ae-e3ab-4484-9e71-4b188c90f8d5-3c128254-b43c-4098-ba8a-6c3866b6a582 state: Requested
342026-08-19T03:19:31.972ZINFOcrucible-agent (worker): applying SMF actions for region 811f03ae-e3ab-4484-9e71-4b188c90f8d5 running snapshot 3c128254-b43c-4098-ba8a-6c3866b6a582 (state Requested)...
352026-08-19T03:19:31.972ZINFOcrucible-agent (dropshot): request completed latency_us = 22820 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42578 req_id = 298b9e58-a204-4df7-8770-8df35275f0ea response_code = 200 uri = /crucible/0/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5/snapshots/3c128254-b43c-4098-ba8a-6c3866b6a582/run
362026-08-19T03:19:31.974ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-811f03ae-e3ab-4484-9e71-4b188c90f8d5-3c128254-b43c-4098-ba8a-6c3866b6a582
372026-08-19T03:19:31.975ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-811f03ae-e3ab-4484-9e71-4b188c90f8d5-3c128254-b43c-4098-ba8a-6c3866b6a582
382026-08-19T03:19:31.981ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-811f03ae-e3ab-4484-9e71-4b188c90f8d5-3c128254-b43c-4098-ba8a-6c3866b6a582
392026-08-19T03:19:31.981ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5/.zfs/snapshot/3c128254-b43c-4098-ba8a-6c3866b6a582
402026-08-19T03:19:31.981ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-08-19T03:19:31.981ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-08-19T03:19:31.981ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
432026-08-19T03:19:31.981ZINFOcrucible-agent (worker): commit
442026-08-19T03:19:31.983ZINFOcrucible-agent (worker): ok!
452026-08-19T03:19:31.986ZINFOcrucible-agent (worker): SMF ok!
462026-08-19T03:19:31.986ZINFOcrucible-agent (datafile): region 811f03ae-e3ab-4484-9e71-4b188c90f8d5 running snapshot 3c128254-b43c-4098-ba8a-6c3866b6a582 state: Requested -> Created
472026-08-19T03:19:32.314ZINFOcrucible-agent (datafile): path is "/data/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5"
482026-08-19T03:19:32.321ZINFOcrucible-agent (datafile): dataset is oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5
492026-08-19T03:19:32.328ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5@3c128254-b43c-4098-ba8a-6c3866b6a582
502026-08-19T03:19:32.328ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_92633a23-3be5-490d-bfcb-5518b4d73013/crucible/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5", "3c128254-b43c-4098-ba8a-6c3866b6a582"]
512026-08-19T03:19:32.333ZINFOcrucible-agent (dropshot): request completed latency_us = 19242 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42578 req_id = 4510723e-9ca4-4256-95d4-bc587594a767 response_code = 200 uri = /crucible/0/regions/811f03ae-e3ab-4484-9e71-4b188c90f8d5/snapshots/3c128254-b43c-4098-ba8a-6c3866b6a582/run