Max level shown:
12026-02-17T00:17:13.561ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:42407
22026-02-17T00:17:13.561ZINFOcrucible-agent (datafile): region c9da3d04-92b6-4ed7-b7d0-0d11dd38a259 state: Requested
32026-02-17T00:17:13.561ZINFOcrucible-agent (dropshot): request completed latency_us = 612 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42407 req_id = a230bf2b-6499-4772-88ce-3a8fb0ef03c3 response_code = 200 uri = /crucible/0/regions
42026-02-17T00:17:13.561ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-02-17T00:17:13.570ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/c9da3d04-92b6-4ed7-b7d0-0d11dd38a259
62026-02-17T00:17:13.570ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/c9da3d04-92b6-4ed7-b7d0-0d11dd38a259
72026-02-17T00:17:13.626ZINFOcrucible-agent (worker): creating region Region { id: RegionId("c9da3d04-92b6-4ed7-b7d0-0d11dd38a259"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/c9da3d04-92b6-4ed7-b7d0-0d11dd38a259" region = c9da3d04-92b6-4ed7-b7d0-0d11dd38a259
82026-02-17T00:17:13.626ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "c9da3d04-92b6-4ed7-b7d0-0d11dd38a259" "--data" "/data/regions/c9da3d04-92b6-4ed7-b7d0-0d11dd38a259" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = c9da3d04-92b6-4ed7-b7d0-0d11dd38a259
92026-02-17T00:17:13.704ZINFOcrucible-agent (worker): region files created ok region = c9da3d04-92b6-4ed7-b7d0-0d11dd38a259
102026-02-17T00:17:13.704ZINFOcrucible-agent (datafile): region c9da3d04-92b6-4ed7-b7d0-0d11dd38a259 state: Requested -> Created
112026-02-17T00:17:13.704ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-02-17T00:17:13.723ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-c9da3d04-92b6-4ed7-b7d0-0d11dd38a259
132026-02-17T00:17:13.734ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-c9da3d04-92b6-4ed7-b7d0-0d11dd38a259
142026-02-17T00:17:13.745ZINFOcrucible-agent (worker): creating config property group
152026-02-17T00:17:13.748ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-c9da3d04-92b6-4ed7-b7d0-0d11dd38a259
162026-02-17T00:17:13.749ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/c9da3d04-92b6-4ed7-b7d0-0d11dd38a259
172026-02-17T00:17:13.749ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-02-17T00:17:13.749ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
192026-02-17T00:17:13.749ZINFOcrucible-agent (worker): commit
202026-02-17T00:17:13.752ZINFOcrucible-agent (worker): ok!
212026-02-17T00:17:13.757ZINFOcrucible-agent (worker): SMF ok!
222026-02-17T00:17:13.829ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42407 req_id = bf73f5e3-e0d9-43a6-a97b-24290aeee6bf response_code = 200 uri = /crucible/0/regions
232026-02-17T00:17:15.103ZINFOcrucible-agent (dropshot): request completed latency_us = 50 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:42407 req_id = 077ce90e-a2b6-4a10-a7d4-11c06d9079d1 response_code = 200 uri = /crucible/0/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719
242026-02-17T00:17:15.107ZINFOcrucible-agent (datafile): path is "/data/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719"
252026-02-17T00:17:15.113ZINFOcrucible-agent (datafile): dataset is oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719
262026-02-17T00:17:15.120ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719@1a3bb064-efc3-447d-afb0-9abe182d5148
272026-02-17T00:17:15.120ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719", "1a3bb064-efc3-447d-afb0-9abe182d5148"]
282026-02-17T00:17:15.128ZINFOcrucible-agent (dropshot): request completed latency_us = 21867 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:42407 req_id = e8e3468e-da12-4145-9335-5e676c722d66 response_code = 200 uri = /crucible/0/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719/snapshots/1a3bb064-efc3-447d-afb0-9abe182d5148
292026-02-17T00:17:15.132ZINFOcrucible-agent (datafile): path is "/data/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719"
302026-02-17T00:17:15.138ZINFOcrucible-agent (datafile): dataset is oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719
312026-02-17T00:17:15.144ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719@1a3bb064-efc3-447d-afb0-9abe182d5148
322026-02-17T00:17:15.144ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719", "1a3bb064-efc3-447d-afb0-9abe182d5148"]
332026-02-17T00:17:15.152ZINFOcrucible-agent (datafile): requesting running snapshot 8ea3b46d-1043-4f01-8ac8-9196f689e719-1a3bb064-efc3-447d-afb0-9abe182d5148 state: Requested
342026-02-17T00:17:15.152ZINFOcrucible-agent (dropshot): request completed latency_us = 20020 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42407 req_id = 75dac6d1-4220-4720-b20d-127fd19e9c58 response_code = 200 uri = /crucible/0/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719/snapshots/1a3bb064-efc3-447d-afb0-9abe182d5148/run
352026-02-17T00:17:15.152ZINFOcrucible-agent (worker): applying SMF actions for region 8ea3b46d-1043-4f01-8ac8-9196f689e719 running snapshot 1a3bb064-efc3-447d-afb0-9abe182d5148 (state Requested)...
362026-02-17T00:17:15.157ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-8ea3b46d-1043-4f01-8ac8-9196f689e719-1a3bb064-efc3-447d-afb0-9abe182d5148
372026-02-17T00:17:15.165ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-8ea3b46d-1043-4f01-8ac8-9196f689e719-1a3bb064-efc3-447d-afb0-9abe182d5148
382026-02-17T00:17:15.175ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-8ea3b46d-1043-4f01-8ac8-9196f689e719-1a3bb064-efc3-447d-afb0-9abe182d5148
392026-02-17T00:17:15.175ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719/.zfs/snapshot/1a3bb064-efc3-447d-afb0-9abe182d5148
402026-02-17T00:17:15.175ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-02-17T00:17:15.175ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-02-17T00:17:15.175ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
432026-02-17T00:17:15.175ZINFOcrucible-agent (worker): commit
442026-02-17T00:17:15.179ZINFOcrucible-agent (worker): ok!
452026-02-17T00:17:15.183ZINFOcrucible-agent (worker): SMF ok!
462026-02-17T00:17:15.183ZINFOcrucible-agent (datafile): region 8ea3b46d-1043-4f01-8ac8-9196f689e719 running snapshot 1a3bb064-efc3-447d-afb0-9abe182d5148 state: Requested -> Created
472026-02-17T00:17:15.297ZINFOcrucible-agent (datafile): path is "/data/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719"
482026-02-17T00:17:15.303ZINFOcrucible-agent (datafile): dataset is oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719
492026-02-17T00:17:15.308ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719@1a3bb064-efc3-447d-afb0-9abe182d5148
502026-02-17T00:17:15.308ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_4777a5eb-a578-4f37-9bc7-cc99247224f3/crucible/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719", "1a3bb064-efc3-447d-afb0-9abe182d5148"]
512026-02-17T00:17:15.315ZINFOcrucible-agent (dropshot): request completed latency_us = 18594 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42407 req_id = 0cbd4182-d04e-407a-aaa1-5cff3a85c93a response_code = 200 uri = /crucible/0/regions/8ea3b46d-1043-4f01-8ac8-9196f689e719/snapshots/1a3bb064-efc3-447d-afb0-9abe182d5148/run