Max level shown:
12026-02-27T03:16:05.309ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:46096
22026-02-27T03:16:05.309ZINFOcrucible-agent (datafile): region 7ad15aef-21e2-4af3-8810-ca503338d31c state: Requested
32026-02-27T03:16:05.310ZINFOcrucible-agent (dropshot): request completed latency_us = 652 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:46096 req_id = f641a68a-8ee4-4d77-94e7-6ce2b562a96f response_code = 200 uri = /crucible/0/regions
42026-02-27T03:16:05.310ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-02-27T03:16:05.317ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/7ad15aef-21e2-4af3-8810-ca503338d31c
62026-02-27T03:16:05.317ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/7ad15aef-21e2-4af3-8810-ca503338d31c
72026-02-27T03:16:05.378ZINFOcrucible-agent (worker): creating region Region { id: RegionId("7ad15aef-21e2-4af3-8810-ca503338d31c"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/7ad15aef-21e2-4af3-8810-ca503338d31c" region = 7ad15aef-21e2-4af3-8810-ca503338d31c
82026-02-27T03:16:05.378ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "7ad15aef-21e2-4af3-8810-ca503338d31c" "--data" "/data/regions/7ad15aef-21e2-4af3-8810-ca503338d31c" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 7ad15aef-21e2-4af3-8810-ca503338d31c
92026-02-27T03:16:05.454ZINFOcrucible-agent (worker): region files created ok region = 7ad15aef-21e2-4af3-8810-ca503338d31c
102026-02-27T03:16:05.454ZINFOcrucible-agent (datafile): region 7ad15aef-21e2-4af3-8810-ca503338d31c state: Requested -> Created
112026-02-27T03:16:05.454ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-02-27T03:16:05.455ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-7ad15aef-21e2-4af3-8810-ca503338d31c
132026-02-27T03:16:05.470ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-7ad15aef-21e2-4af3-8810-ca503338d31c
142026-02-27T03:16:05.478ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:46096 req_id = 7a880923-bb63-4bb3-ba40-18c14369a223 response_code = 200 uri = /crucible/0/regions
152026-02-27T03:16:05.494ZINFOcrucible-agent (worker): creating config property group
162026-02-27T03:16:05.497ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-7ad15aef-21e2-4af3-8810-ca503338d31c
172026-02-27T03:16:05.498ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/7ad15aef-21e2-4af3-8810-ca503338d31c
182026-02-27T03:16:05.498ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192026-02-27T03:16:05.498ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
202026-02-27T03:16:05.498ZINFOcrucible-agent (worker): commit
212026-02-27T03:16:05.501ZINFOcrucible-agent (worker): ok!
222026-02-27T03:16:05.516ZINFOcrucible-agent (worker): SMF ok!
232026-02-27T03:16:06.411ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:46096 req_id = 9415dd10-618c-4d08-9018-8c02317c1210 response_code = 200 uri = /crucible/0/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec
242026-02-27T03:16:06.415ZINFOcrucible-agent (datafile): path is "/data/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec"
252026-02-27T03:16:06.422ZINFOcrucible-agent (datafile): dataset is oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec
262026-02-27T03:16:06.428ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec@344cbb68-bf46-449c-890b-59b44b111c0c
272026-02-27T03:16:06.428ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec", "344cbb68-bf46-449c-890b-59b44b111c0c"]
282026-02-27T03:16:06.436ZINFOcrucible-agent (dropshot): request completed latency_us = 20977 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:46096 req_id = ed49251c-578e-40d5-861d-07f99827d025 response_code = 200 uri = /crucible/0/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec/snapshots/344cbb68-bf46-449c-890b-59b44b111c0c
292026-02-27T03:16:06.440ZINFOcrucible-agent (datafile): path is "/data/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec"
302026-02-27T03:16:06.446ZINFOcrucible-agent (datafile): dataset is oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec
312026-02-27T03:16:06.452ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec@344cbb68-bf46-449c-890b-59b44b111c0c
322026-02-27T03:16:06.452ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec", "344cbb68-bf46-449c-890b-59b44b111c0c"]
332026-02-27T03:16:06.459ZINFOcrucible-agent (datafile): requesting running snapshot fc79dce4-ecaa-4c39-ad80-ef0c024709ec-344cbb68-bf46-449c-890b-59b44b111c0c state: Requested
342026-02-27T03:16:06.459ZINFOcrucible-agent (dropshot): request completed latency_us = 19299 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:46096 req_id = cf9f78f6-9585-43a4-a232-3d7ef648e4a2 response_code = 200 uri = /crucible/0/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec/snapshots/344cbb68-bf46-449c-890b-59b44b111c0c/run
352026-02-27T03:16:06.459ZINFOcrucible-agent (worker): applying SMF actions for region fc79dce4-ecaa-4c39-ad80-ef0c024709ec running snapshot 344cbb68-bf46-449c-890b-59b44b111c0c (state Requested)...
362026-02-27T03:16:06.464ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-fc79dce4-ecaa-4c39-ad80-ef0c024709ec-344cbb68-bf46-449c-890b-59b44b111c0c
372026-02-27T03:16:06.472ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-fc79dce4-ecaa-4c39-ad80-ef0c024709ec-344cbb68-bf46-449c-890b-59b44b111c0c
382026-02-27T03:16:06.481ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-fc79dce4-ecaa-4c39-ad80-ef0c024709ec-344cbb68-bf46-449c-890b-59b44b111c0c
392026-02-27T03:16:06.482ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec/.zfs/snapshot/344cbb68-bf46-449c-890b-59b44b111c0c
402026-02-27T03:16:06.482ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-02-27T03:16:06.482ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-02-27T03:16:06.482ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432026-02-27T03:16:06.482ZINFOcrucible-agent (worker): commit
442026-02-27T03:16:06.485ZINFOcrucible-agent (worker): ok!
452026-02-27T03:16:06.489ZINFOcrucible-agent (worker): SMF ok!
462026-02-27T03:16:06.489ZINFOcrucible-agent (datafile): region fc79dce4-ecaa-4c39-ad80-ef0c024709ec running snapshot 344cbb68-bf46-449c-890b-59b44b111c0c state: Requested -> Created
472026-02-27T03:16:06.720ZINFOcrucible-agent (datafile): path is "/data/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec"
482026-02-27T03:16:06.729ZINFOcrucible-agent (datafile): dataset is oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec
492026-02-27T03:16:06.738ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec@344cbb68-bf46-449c-890b-59b44b111c0c
502026-02-27T03:16:06.738ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_34ec0d6a-e024-4aea-b864-d44b71bdb661/crucible/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec", "344cbb68-bf46-449c-890b-59b44b111c0c"]
512026-02-27T03:16:06.748ZINFOcrucible-agent (dropshot): request completed latency_us = 27978 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:46096 req_id = 070d38ec-ff7b-4ef2-9f98-acd16b136175 response_code = 200 uri = /crucible/0/regions/fc79dce4-ecaa-4c39-ad80-ef0c024709ec/snapshots/344cbb68-bf46-449c-890b-59b44b111c0c/run