Max level shown:
12026-10-14T01:40:58.524ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:38660
22026-10-14T01:40:58.524ZINFOcrucible-agent (datafile): region cd90db74-ceb2-433f-83a7-fe4f8c6a91c8 state: Requested
32026-10-14T01:40:58.524ZINFOcrucible-agent (dropshot): request completed latency_us = 691 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38660 req_id = 4400ab7f-5c6c-416d-a2a4-b36a6c8ccb6c response_code = 200 uri = /crucible/0/regions
42026-10-14T01:40:58.524ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-10-14T01:40:58.533ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/cd90db74-ceb2-433f-83a7-fe4f8c6a91c8
62026-10-14T01:40:58.533ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/cd90db74-ceb2-433f-83a7-fe4f8c6a91c8
72026-10-14T01:40:58.582ZINFOcrucible-agent (worker): creating region Region { id: RegionId("cd90db74-ceb2-433f-83a7-fe4f8c6a91c8"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/cd90db74-ceb2-433f-83a7-fe4f8c6a91c8" region = cd90db74-ceb2-433f-83a7-fe4f8c6a91c8
82026-10-14T01:40:58.582ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "cd90db74-ceb2-433f-83a7-fe4f8c6a91c8" "--data" "/data/regions/cd90db74-ceb2-433f-83a7-fe4f8c6a91c8" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = cd90db74-ceb2-433f-83a7-fe4f8c6a91c8
92026-10-14T01:40:58.664ZINFOcrucible-agent (worker): region files created ok region = cd90db74-ceb2-433f-83a7-fe4f8c6a91c8
102026-10-14T01:40:58.664ZINFOcrucible-agent (datafile): region cd90db74-ceb2-433f-83a7-fe4f8c6a91c8 state: Requested -> Created
112026-10-14T01:40:58.664ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-10-14T01:40:58.675ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-cd90db74-ceb2-433f-83a7-fe4f8c6a91c8
132026-10-14T01:40:58.677ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38660 req_id = 30e0ff92-7b3c-41d9-aad6-4ed6d653364d response_code = 200 uri = /crucible/0/regions
142026-10-14T01:40:58.679ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-cd90db74-ceb2-433f-83a7-fe4f8c6a91c8
152026-10-14T01:40:58.694ZINFOcrucible-agent (worker): creating config property group
162026-10-14T01:40:58.698ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-cd90db74-ceb2-433f-83a7-fe4f8c6a91c8
172026-10-14T01:40:58.698ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/cd90db74-ceb2-433f-83a7-fe4f8c6a91c8
182026-10-14T01:40:58.698ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192026-10-14T01:40:58.698ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
202026-10-14T01:40:58.698ZINFOcrucible-agent (worker): commit
212026-10-14T01:40:58.701ZINFOcrucible-agent (worker): ok!
222026-10-14T01:40:58.706ZINFOcrucible-agent (worker): SMF ok!
232026-10-14T01:41:00.243ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38660 req_id = 2391c4b4-8bba-4148-a3d3-69ef37d62e34 response_code = 200 uri = /crucible/0/regions/6e366316-ed98-4653-9a74-ab9be612c9a6
242026-10-14T01:41:00.250ZINFOcrucible-agent (datafile): path is "/data/regions/6e366316-ed98-4653-9a74-ab9be612c9a6"
252026-10-14T01:41:00.257ZINFOcrucible-agent (datafile): dataset is oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/6e366316-ed98-4653-9a74-ab9be612c9a6
262026-10-14T01:41:00.262ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/6e366316-ed98-4653-9a74-ab9be612c9a6@67404747-1fa4-4031-844e-a8d15035fdca
272026-10-14T01:41:00.262ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/6e366316-ed98-4653-9a74-ab9be612c9a6", "67404747-1fa4-4031-844e-a8d15035fdca"]
282026-10-14T01:41:00.272ZINFOcrucible-agent (dropshot): request completed latency_us = 25633 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38660 req_id = 5b4886de-34b7-416b-ae30-2a1dab5974d4 response_code = 200 uri = /crucible/0/regions/6e366316-ed98-4653-9a74-ab9be612c9a6/snapshots/67404747-1fa4-4031-844e-a8d15035fdca
292026-10-14T01:41:00.275ZINFOcrucible-agent (datafile): path is "/data/regions/6e366316-ed98-4653-9a74-ab9be612c9a6"
302026-10-14T01:41:00.281ZINFOcrucible-agent (datafile): dataset is oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/6e366316-ed98-4653-9a74-ab9be612c9a6
312026-10-14T01:41:00.288ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/6e366316-ed98-4653-9a74-ab9be612c9a6@67404747-1fa4-4031-844e-a8d15035fdca
322026-10-14T01:41:00.288ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/6e366316-ed98-4653-9a74-ab9be612c9a6", "67404747-1fa4-4031-844e-a8d15035fdca"]
332026-10-14T01:41:00.294ZINFOcrucible-agent (datafile): requesting running snapshot 6e366316-ed98-4653-9a74-ab9be612c9a6-67404747-1fa4-4031-844e-a8d15035fdca state: Requested
342026-10-14T01:41:00.294ZINFOcrucible-agent (dropshot): request completed latency_us = 19701 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38660 req_id = 35ede9aa-6b25-4df9-a0f4-aa80461371e3 response_code = 200 uri = /crucible/0/regions/6e366316-ed98-4653-9a74-ab9be612c9a6/snapshots/67404747-1fa4-4031-844e-a8d15035fdca/run
352026-10-14T01:41:00.294ZINFOcrucible-agent (worker): applying SMF actions for region 6e366316-ed98-4653-9a74-ab9be612c9a6 running snapshot 67404747-1fa4-4031-844e-a8d15035fdca (state Requested)...
362026-10-14T01:41:00.297ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-6e366316-ed98-4653-9a74-ab9be612c9a6-67404747-1fa4-4031-844e-a8d15035fdca
372026-10-14T01:41:00.302ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-6e366316-ed98-4653-9a74-ab9be612c9a6-67404747-1fa4-4031-844e-a8d15035fdca
382026-10-14T01:41:00.311ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-6e366316-ed98-4653-9a74-ab9be612c9a6-67404747-1fa4-4031-844e-a8d15035fdca
392026-10-14T01:41:00.312ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/6e366316-ed98-4653-9a74-ab9be612c9a6/.zfs/snapshot/67404747-1fa4-4031-844e-a8d15035fdca
402026-10-14T01:41:00.312ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-10-14T01:41:00.312ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-10-14T01:41:00.312ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
432026-10-14T01:41:00.312ZINFOcrucible-agent (worker): commit
442026-10-14T01:41:00.315ZINFOcrucible-agent (worker): ok!
452026-10-14T01:41:00.320ZINFOcrucible-agent (worker): SMF ok!
462026-10-14T01:41:00.320ZINFOcrucible-agent (datafile): region 6e366316-ed98-4653-9a74-ab9be612c9a6 running snapshot 67404747-1fa4-4031-844e-a8d15035fdca state: Requested -> Created
472026-10-14T01:41:00.595ZINFOcrucible-agent (datafile): path is "/data/regions/6e366316-ed98-4653-9a74-ab9be612c9a6"
482026-10-14T01:41:00.601ZINFOcrucible-agent (datafile): dataset is oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/6e366316-ed98-4653-9a74-ab9be612c9a6
492026-10-14T01:41:00.606ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/6e366316-ed98-4653-9a74-ab9be612c9a6@67404747-1fa4-4031-844e-a8d15035fdca
502026-10-14T01:41:00.606ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_092244ff-277f-4ddd-a58f-23dfc474a683/crucible/regions/6e366316-ed98-4653-9a74-ab9be612c9a6", "67404747-1fa4-4031-844e-a8d15035fdca"]
512026-10-14T01:41:00.610ZINFOcrucible-agent (dropshot): request completed latency_us = 15033 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38660 req_id = 19e9fa88-ec53-442c-a628-60c8014d748d response_code = 200 uri = /crucible/0/regions/6e366316-ed98-4653-9a74-ab9be612c9a6/snapshots/67404747-1fa4-4031-844e-a8d15035fdca/run