Max level shown:
12026-05-08T22:48:41.524ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::c]:56304
22026-05-08T22:48:41.524ZINFOcrucible-agent (datafile): region d5a48bda-ff60-4f34-b285-9765b1ad1de3 state: Requested
32026-05-08T22:48:41.524ZINFOcrucible-agent (dropshot): request completed latency_us = 757 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:56304 req_id = 51373747-0384-4299-b7e3-c38f8d0511e8 response_code = 200 uri = /crucible/0/regions
42026-05-08T22:48:41.524ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-05-08T22:48:41.535ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/d5a48bda-ff60-4f34-b285-9765b1ad1de3
62026-05-08T22:48:41.535ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/d5a48bda-ff60-4f34-b285-9765b1ad1de3
72026-05-08T22:48:41.587ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d5a48bda-ff60-4f34-b285-9765b1ad1de3"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d5a48bda-ff60-4f34-b285-9765b1ad1de3" region = d5a48bda-ff60-4f34-b285-9765b1ad1de3
82026-05-08T22:48:41.587ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d5a48bda-ff60-4f34-b285-9765b1ad1de3" "--data" "/data/regions/d5a48bda-ff60-4f34-b285-9765b1ad1de3" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = d5a48bda-ff60-4f34-b285-9765b1ad1de3
92026-05-08T22:48:41.701ZINFOcrucible-agent (worker): region files created ok region = d5a48bda-ff60-4f34-b285-9765b1ad1de3
102026-05-08T22:48:41.701ZINFOcrucible-agent (datafile): region d5a48bda-ff60-4f34-b285-9765b1ad1de3 state: Requested -> Created
112026-05-08T22:48:41.701ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-05-08T22:48:41.702ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d5a48bda-ff60-4f34-b285-9765b1ad1de3
132026-05-08T22:48:41.712ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d5a48bda-ff60-4f34-b285-9765b1ad1de3
142026-05-08T22:48:41.734ZINFOcrucible-agent (worker): creating config property group
152026-05-08T22:48:41.737ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d5a48bda-ff60-4f34-b285-9765b1ad1de3
162026-05-08T22:48:41.738ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d5a48bda-ff60-4f34-b285-9765b1ad1de3
172026-05-08T22:48:41.738ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-05-08T22:48:41.738ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
192026-05-08T22:48:41.738ZINFOcrucible-agent (worker): commit
202026-05-08T22:48:41.741ZINFOcrucible-agent (worker): ok!
212026-05-08T22:48:41.757ZINFOcrucible-agent (worker): SMF ok!
222026-05-08T22:48:41.769ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:56304 req_id = 4f83e005-83a9-4b9b-aa66-e297952e6635 response_code = 200 uri = /crucible/0/regions
232026-05-08T22:48:43.597ZINFOcrucible-agent (dropshot): request completed latency_us = 81 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:56304 req_id = 2310685f-7253-4875-a093-af53cda1fbcd response_code = 200 uri = /crucible/0/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc
242026-05-08T22:48:43.600ZINFOcrucible-agent (datafile): path is "/data/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc"
252026-05-08T22:48:43.606ZINFOcrucible-agent (datafile): dataset is oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc
262026-05-08T22:48:43.612ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc@493c36cd-977b-4b73-a041-c03a32f6d5de
272026-05-08T22:48:43.612ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc", "493c36cd-977b-4b73-a041-c03a32f6d5de"]
282026-05-08T22:48:43.622ZINFOcrucible-agent (dropshot): request completed latency_us = 22035 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:56304 req_id = ab1d7165-bb9e-4afe-b5dd-ab33c60d1e27 response_code = 200 uri = /crucible/0/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc/snapshots/493c36cd-977b-4b73-a041-c03a32f6d5de
292026-05-08T22:48:43.626ZINFOcrucible-agent (datafile): path is "/data/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc"
302026-05-08T22:48:43.632ZINFOcrucible-agent (datafile): dataset is oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc
312026-05-08T22:48:43.637ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc@493c36cd-977b-4b73-a041-c03a32f6d5de
322026-05-08T22:48:43.637ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc", "493c36cd-977b-4b73-a041-c03a32f6d5de"]
332026-05-08T22:48:43.646ZINFOcrucible-agent (datafile): requesting running snapshot eb05b544-73b4-429f-bdd2-534fe8005bfc-493c36cd-977b-4b73-a041-c03a32f6d5de state: Requested
342026-05-08T22:48:43.646ZINFOcrucible-agent (dropshot): request completed latency_us = 20508 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:56304 req_id = 1591a220-dc83-47ad-b888-57fc17ded940 response_code = 200 uri = /crucible/0/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc/snapshots/493c36cd-977b-4b73-a041-c03a32f6d5de/run
352026-05-08T22:48:43.646ZINFOcrucible-agent (worker): applying SMF actions for region eb05b544-73b4-429f-bdd2-534fe8005bfc running snapshot 493c36cd-977b-4b73-a041-c03a32f6d5de (state Requested)...
362026-05-08T22:48:43.650ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-eb05b544-73b4-429f-bdd2-534fe8005bfc-493c36cd-977b-4b73-a041-c03a32f6d5de
372026-05-08T22:48:43.656ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-eb05b544-73b4-429f-bdd2-534fe8005bfc-493c36cd-977b-4b73-a041-c03a32f6d5de
382026-05-08T22:48:43.667ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-eb05b544-73b4-429f-bdd2-534fe8005bfc-493c36cd-977b-4b73-a041-c03a32f6d5de
392026-05-08T22:48:43.667ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc/.zfs/snapshot/493c36cd-977b-4b73-a041-c03a32f6d5de
402026-05-08T22:48:43.667ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-05-08T22:48:43.667ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-05-08T22:48:43.667ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
432026-05-08T22:48:43.667ZINFOcrucible-agent (worker): commit
442026-05-08T22:48:43.671ZINFOcrucible-agent (worker): ok!
452026-05-08T22:48:43.676ZINFOcrucible-agent (worker): SMF ok!
462026-05-08T22:48:43.676ZINFOcrucible-agent (datafile): region eb05b544-73b4-429f-bdd2-534fe8005bfc running snapshot 493c36cd-977b-4b73-a041-c03a32f6d5de state: Requested -> Created
472026-05-08T22:48:44.000ZINFOcrucible-agent (datafile): path is "/data/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc"
482026-05-08T22:48:44.007ZINFOcrucible-agent (datafile): dataset is oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc
492026-05-08T22:48:44.014ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc@493c36cd-977b-4b73-a041-c03a32f6d5de
502026-05-08T22:48:44.014ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f1dd5c7c-de43-4108-865f-a09d1713c40b/crucible/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc", "493c36cd-977b-4b73-a041-c03a32f6d5de"]
512026-05-08T22:48:44.021ZINFOcrucible-agent (dropshot): request completed latency_us = 21604 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:56304 req_id = 7d990d6e-9389-481d-8968-4548a4c971f4 response_code = 200 uri = /crucible/0/regions/eb05b544-73b4-429f-bdd2-534fe8005bfc/snapshots/493c36cd-977b-4b73-a041-c03a32f6d5de/run