Max level shown:
12026-04-30T01:08:43.623ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:50999
22026-04-30T01:08:43.623ZINFOcrucible-agent (datafile): region a94ec46a-8ca8-4679-b386-b93ce654e3bb state: Requested
32026-04-30T01:08:43.624ZINFOcrucible-agent (dropshot): request completed latency_us = 663 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50999 req_id = fe212c0f-cf80-4640-b3ad-000e955b78e1 response_code = 200 uri = /crucible/0/regions
42026-04-30T01:08:43.624ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-04-30T01:08:43.634ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/a94ec46a-8ca8-4679-b386-b93ce654e3bb
62026-04-30T01:08:43.634ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/a94ec46a-8ca8-4679-b386-b93ce654e3bb
72026-04-30T01:08:43.688ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a94ec46a-8ca8-4679-b386-b93ce654e3bb"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a94ec46a-8ca8-4679-b386-b93ce654e3bb" region = a94ec46a-8ca8-4679-b386-b93ce654e3bb
82026-04-30T01:08:43.688ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a94ec46a-8ca8-4679-b386-b93ce654e3bb" "--data" "/data/regions/a94ec46a-8ca8-4679-b386-b93ce654e3bb" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = a94ec46a-8ca8-4679-b386-b93ce654e3bb
92026-04-30T01:08:43.775ZINFOcrucible-agent (worker): region files created ok region = a94ec46a-8ca8-4679-b386-b93ce654e3bb
102026-04-30T01:08:43.775ZINFOcrucible-agent (datafile): region a94ec46a-8ca8-4679-b386-b93ce654e3bb state: Requested -> Created
112026-04-30T01:08:43.775ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-04-30T01:08:43.795ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a94ec46a-8ca8-4679-b386-b93ce654e3bb
132026-04-30T01:08:43.799ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a94ec46a-8ca8-4679-b386-b93ce654e3bb
142026-04-30T01:08:43.808ZINFOcrucible-agent (worker): creating config property group
152026-04-30T01:08:43.812ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a94ec46a-8ca8-4679-b386-b93ce654e3bb
162026-04-30T01:08:43.812ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a94ec46a-8ca8-4679-b386-b93ce654e3bb
172026-04-30T01:08:43.812ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-04-30T01:08:43.812ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192026-04-30T01:08:43.812ZINFOcrucible-agent (worker): commit
202026-04-30T01:08:43.816ZINFOcrucible-agent (worker): ok!
212026-04-30T01:08:43.820ZINFOcrucible-agent (worker): SMF ok!
222026-04-30T01:08:43.905ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50999 req_id = 0640e665-253a-49f8-9b60-a31798365810 response_code = 200 uri = /crucible/0/regions
232026-04-30T01:08:45.452ZINFOcrucible-agent (dropshot): request completed latency_us = 67 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:50999 req_id = 0d051929-b2ab-45a1-afb9-903a4de3b1f5 response_code = 200 uri = /crucible/0/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9
242026-04-30T01:08:45.456ZINFOcrucible-agent (datafile): path is "/data/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9"
252026-04-30T01:08:45.462ZINFOcrucible-agent (datafile): dataset is oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9
262026-04-30T01:08:45.466ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9@05e17244-be4b-4494-a914-09def1d38aeb
272026-04-30T01:08:45.466ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9", "05e17244-be4b-4494-a914-09def1d38aeb"]
282026-04-30T01:08:45.475ZINFOcrucible-agent (dropshot): request completed latency_us = 19544 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:50999 req_id = 85c7f939-42f1-43fe-8e7a-fc9f65e38f7b response_code = 200 uri = /crucible/0/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9/snapshots/05e17244-be4b-4494-a914-09def1d38aeb
292026-04-30T01:08:45.479ZINFOcrucible-agent (datafile): path is "/data/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9"
302026-04-30T01:08:45.485ZINFOcrucible-agent (datafile): dataset is oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9
312026-04-30T01:08:45.490ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9@05e17244-be4b-4494-a914-09def1d38aeb
322026-04-30T01:08:45.490ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9", "05e17244-be4b-4494-a914-09def1d38aeb"]
332026-04-30T01:08:45.500ZINFOcrucible-agent (datafile): requesting running snapshot 1ddaef0f-6645-447d-9d83-92aba2c203a9-05e17244-be4b-4494-a914-09def1d38aeb state: Requested
342026-04-30T01:08:45.500ZINFOcrucible-agent (dropshot): request completed latency_us = 21156 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50999 req_id = b848b1de-f13a-4eb6-9aab-b6b217a7a42e response_code = 200 uri = /crucible/0/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9/snapshots/05e17244-be4b-4494-a914-09def1d38aeb/run
352026-04-30T01:08:45.500ZINFOcrucible-agent (worker): applying SMF actions for region 1ddaef0f-6645-447d-9d83-92aba2c203a9 running snapshot 05e17244-be4b-4494-a914-09def1d38aeb (state Requested)...
362026-04-30T01:08:45.504ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-1ddaef0f-6645-447d-9d83-92aba2c203a9-05e17244-be4b-4494-a914-09def1d38aeb
372026-04-30T01:08:45.512ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-1ddaef0f-6645-447d-9d83-92aba2c203a9-05e17244-be4b-4494-a914-09def1d38aeb
382026-04-30T01:08:45.522ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-1ddaef0f-6645-447d-9d83-92aba2c203a9-05e17244-be4b-4494-a914-09def1d38aeb
392026-04-30T01:08:45.522ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9/.zfs/snapshot/05e17244-be4b-4494-a914-09def1d38aeb
402026-04-30T01:08:45.522ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-04-30T01:08:45.522ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-04-30T01:08:45.522ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432026-04-30T01:08:45.522ZINFOcrucible-agent (worker): commit
442026-04-30T01:08:45.525ZINFOcrucible-agent (worker): ok!
452026-04-30T01:08:45.530ZINFOcrucible-agent (worker): SMF ok!
462026-04-30T01:08:45.530ZINFOcrucible-agent (datafile): region 1ddaef0f-6645-447d-9d83-92aba2c203a9 running snapshot 05e17244-be4b-4494-a914-09def1d38aeb state: Requested -> Created
472026-04-30T01:08:45.775ZINFOcrucible-agent (datafile): path is "/data/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9"
482026-04-30T01:08:45.782ZINFOcrucible-agent (datafile): dataset is oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9
492026-04-30T01:08:45.787ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9@05e17244-be4b-4494-a914-09def1d38aeb
502026-04-30T01:08:45.787ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6c9e00fb-9deb-4552-812d-4d99dba8a61f/crucible/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9", "05e17244-be4b-4494-a914-09def1d38aeb"]
512026-04-30T01:08:45.792ZINFOcrucible-agent (dropshot): request completed latency_us = 16553 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50999 req_id = a6059e1e-5d8f-43da-a8fc-a2b8d0f38590 response_code = 200 uri = /crucible/0/regions/1ddaef0f-6645-447d-9d83-92aba2c203a9/snapshots/05e17244-be4b-4494-a914-09def1d38aeb/run