Max level shown:
12026-04-28T19:23:33.733ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::c]:59923
22026-04-28T19:23:33.733ZINFOcrucible-agent (datafile): region bad1b58b-8d11-4bce-96ee-b72781805ab6 state: Requested
32026-04-28T19:23:33.734ZINFOcrucible-agent (dropshot): request completed latency_us = 907 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:59923 req_id = 49d3cfed-6dae-4924-a2e8-b523548d2399 response_code = 200 uri = /crucible/0/regions
42026-04-28T19:23:33.734ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-04-28T19:23:33.741ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/bad1b58b-8d11-4bce-96ee-b72781805ab6
62026-04-28T19:23:33.741ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/bad1b58b-8d11-4bce-96ee-b72781805ab6
72026-04-28T19:23:33.815ZINFOcrucible-agent (worker): creating region Region { id: RegionId("bad1b58b-8d11-4bce-96ee-b72781805ab6"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/bad1b58b-8d11-4bce-96ee-b72781805ab6" region = bad1b58b-8d11-4bce-96ee-b72781805ab6
82026-04-28T19:23:33.815ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "bad1b58b-8d11-4bce-96ee-b72781805ab6" "--data" "/data/regions/bad1b58b-8d11-4bce-96ee-b72781805ab6" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = bad1b58b-8d11-4bce-96ee-b72781805ab6
92026-04-28T19:23:33.891ZINFOcrucible-agent (worker): region files created ok region = bad1b58b-8d11-4bce-96ee-b72781805ab6
102026-04-28T19:23:33.891ZINFOcrucible-agent (datafile): region bad1b58b-8d11-4bce-96ee-b72781805ab6 state: Requested -> Created
112026-04-28T19:23:33.891ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-04-28T19:23:33.912ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-bad1b58b-8d11-4bce-96ee-b72781805ab6
132026-04-28T19:23:33.918ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-bad1b58b-8d11-4bce-96ee-b72781805ab6
142026-04-28T19:23:33.927ZINFOcrucible-agent (worker): creating config property group
152026-04-28T19:23:33.930ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-bad1b58b-8d11-4bce-96ee-b72781805ab6
162026-04-28T19:23:33.930ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/bad1b58b-8d11-4bce-96ee-b72781805ab6
172026-04-28T19:23:33.930ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-04-28T19:23:33.931ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
192026-04-28T19:23:33.931ZINFOcrucible-agent (worker): commit
202026-04-28T19:23:33.934ZINFOcrucible-agent (worker): ok!
212026-04-28T19:23:33.939ZINFOcrucible-agent (worker): SMF ok!
222026-04-28T19:23:34.114ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:59923 req_id = 1267d6c6-da11-4e0b-b240-4a759865dcb5 response_code = 200 uri = /crucible/0/regions
232026-04-28T19:23:34.876ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:59923 req_id = 421a2752-fbaa-465f-9b8c-fa03eb89ab82 response_code = 200 uri = /crucible/0/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a
242026-04-28T19:23:34.879ZINFOcrucible-agent (datafile): path is "/data/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a"
252026-04-28T19:23:34.886ZINFOcrucible-agent (datafile): dataset is oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a
262026-04-28T19:23:34.893ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a@b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b
272026-04-28T19:23:34.893ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a", "b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b"]
282026-04-28T19:23:34.901ZINFOcrucible-agent (dropshot): request completed latency_us = 21934 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:59923 req_id = 042cbf5c-3b54-4890-86e0-64be917bb11a response_code = 200 uri = /crucible/0/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a/snapshots/b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b
292026-04-28T19:23:34.905ZINFOcrucible-agent (datafile): path is "/data/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a"
302026-04-28T19:23:34.912ZINFOcrucible-agent (datafile): dataset is oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a
312026-04-28T19:23:34.918ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a@b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b
322026-04-28T19:23:34.918ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a", "b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b"]
332026-04-28T19:23:34.926ZINFOcrucible-agent (datafile): requesting running snapshot 6b959240-66d9-4a41-baf6-ff10eaf3630a-b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b state: Requested
342026-04-28T19:23:34.926ZINFOcrucible-agent (dropshot): request completed latency_us = 20846 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:59923 req_id = b7aec72e-983a-4b39-b4f5-6420285fa33d response_code = 200 uri = /crucible/0/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a/snapshots/b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b/run
352026-04-28T19:23:34.926ZINFOcrucible-agent (worker): applying SMF actions for region 6b959240-66d9-4a41-baf6-ff10eaf3630a running snapshot b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b (state Requested)...
362026-04-28T19:23:34.930ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-6b959240-66d9-4a41-baf6-ff10eaf3630a-b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b
372026-04-28T19:23:34.939ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-6b959240-66d9-4a41-baf6-ff10eaf3630a-b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b
382026-04-28T19:23:34.949ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-6b959240-66d9-4a41-baf6-ff10eaf3630a-b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b
392026-04-28T19:23:34.949ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a/.zfs/snapshot/b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b
402026-04-28T19:23:34.949ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-04-28T19:23:34.949ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-04-28T19:23:34.949ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432026-04-28T19:23:34.949ZINFOcrucible-agent (worker): commit
442026-04-28T19:23:34.953ZINFOcrucible-agent (worker): ok!
452026-04-28T19:23:34.958ZINFOcrucible-agent (worker): SMF ok!
462026-04-28T19:23:34.958ZINFOcrucible-agent (datafile): region 6b959240-66d9-4a41-baf6-ff10eaf3630a running snapshot b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b state: Requested -> Created
472026-04-28T19:23:35.200ZINFOcrucible-agent (datafile): path is "/data/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a"
482026-04-28T19:23:35.207ZINFOcrucible-agent (datafile): dataset is oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a
492026-04-28T19:23:35.214ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a@b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b
502026-04-28T19:23:35.214ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1d12e54b-958d-48b7-a6bd-534f273023b0/crucible/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a", "b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b"]
512026-04-28T19:23:35.220ZINFOcrucible-agent (dropshot): request completed latency_us = 19760 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:59923 req_id = 7160cd0e-03a2-498f-ab8d-25e444725bdf response_code = 200 uri = /crucible/0/regions/6b959240-66d9-4a41-baf6-ff10eaf3630a/snapshots/b3f5f829-d4b3-48c7-9749-ac8a3f2fc69b/run