Max level shown:
12026-01-22T05:18:17.232ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:57018
22026-01-22T05:18:17.232ZINFOcrucible-agent (datafile): region 8c3fbb02-d516-4f20-82eb-2716cbd4ceb3 state: Requested
32026-01-22T05:18:17.233ZINFOcrucible-agent (dropshot): request completed latency_us = 619 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:57018 req_id = 94f884b0-7783-4281-84ad-91a5651e2d92 response_code = 200 uri = /crucible/0/regions
42026-01-22T05:18:17.233ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-01-22T05:18:17.241ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/8c3fbb02-d516-4f20-82eb-2716cbd4ceb3
62026-01-22T05:18:17.241ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/8c3fbb02-d516-4f20-82eb-2716cbd4ceb3
72026-01-22T05:18:17.289ZINFOcrucible-agent (worker): creating region Region { id: RegionId("8c3fbb02-d516-4f20-82eb-2716cbd4ceb3"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/8c3fbb02-d516-4f20-82eb-2716cbd4ceb3" region = 8c3fbb02-d516-4f20-82eb-2716cbd4ceb3
82026-01-22T05:18:17.289ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "8c3fbb02-d516-4f20-82eb-2716cbd4ceb3" "--data" "/data/regions/8c3fbb02-d516-4f20-82eb-2716cbd4ceb3" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 8c3fbb02-d516-4f20-82eb-2716cbd4ceb3
92026-01-22T05:18:17.405ZINFOcrucible-agent (worker): region files created ok region = 8c3fbb02-d516-4f20-82eb-2716cbd4ceb3
102026-01-22T05:18:17.405ZINFOcrucible-agent (datafile): region 8c3fbb02-d516-4f20-82eb-2716cbd4ceb3 state: Requested -> Created
112026-01-22T05:18:17.405ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-01-22T05:18:17.425ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-8c3fbb02-d516-4f20-82eb-2716cbd4ceb3
132026-01-22T05:18:17.436ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-8c3fbb02-d516-4f20-82eb-2716cbd4ceb3
142026-01-22T05:18:17.445ZINFOcrucible-agent (worker): creating config property group
152026-01-22T05:18:17.448ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8c3fbb02-d516-4f20-82eb-2716cbd4ceb3
162026-01-22T05:18:17.449ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8c3fbb02-d516-4f20-82eb-2716cbd4ceb3
172026-01-22T05:18:17.449ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-01-22T05:18:17.449ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192026-01-22T05:18:17.449ZINFOcrucible-agent (worker): commit
202026-01-22T05:18:17.452ZINFOcrucible-agent (worker): ok!
212026-01-22T05:18:17.457ZINFOcrucible-agent (worker): SMF ok!
222026-01-22T05:18:17.492ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:57018 req_id = be66e8a5-c1b7-4de5-b3d0-d739949c5240 response_code = 200 uri = /crucible/0/regions
232026-01-22T05:18:19.086ZINFOcrucible-agent (dropshot): request completed latency_us = 51 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:57018 req_id = b1715e97-17f8-434f-b609-818535309d2a response_code = 200 uri = /crucible/0/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e
242026-01-22T05:18:19.089ZINFOcrucible-agent (datafile): path is "/data/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e"
252026-01-22T05:18:19.096ZINFOcrucible-agent (datafile): dataset is oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e
262026-01-22T05:18:19.104ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e@80c18f80-1949-4617-b561-695e43820ce1
272026-01-22T05:18:19.104ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e", "80c18f80-1949-4617-b561-695e43820ce1"]
282026-01-22T05:18:19.112ZINFOcrucible-agent (dropshot): request completed latency_us = 23179 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:57018 req_id = 121e3ad6-9867-4a4e-8b20-7d63eb97f571 response_code = 200 uri = /crucible/0/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e/snapshots/80c18f80-1949-4617-b561-695e43820ce1
292026-01-22T05:18:19.115ZINFOcrucible-agent (datafile): path is "/data/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e"
302026-01-22T05:18:19.122ZINFOcrucible-agent (datafile): dataset is oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e
312026-01-22T05:18:19.129ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e@80c18f80-1949-4617-b561-695e43820ce1
322026-01-22T05:18:19.129ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e", "80c18f80-1949-4617-b561-695e43820ce1"]
332026-01-22T05:18:19.136ZINFOcrucible-agent (datafile): requesting running snapshot 5b8d09a6-1708-4385-8ad0-c80611419d6e-80c18f80-1949-4617-b561-695e43820ce1 state: Requested
342026-01-22T05:18:19.136ZINFOcrucible-agent (dropshot): request completed latency_us = 20968 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:57018 req_id = a65a8197-af31-4c7b-b4bd-801b5e61be80 response_code = 200 uri = /crucible/0/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e/snapshots/80c18f80-1949-4617-b561-695e43820ce1/run
352026-01-22T05:18:19.136ZINFOcrucible-agent (worker): applying SMF actions for region 5b8d09a6-1708-4385-8ad0-c80611419d6e running snapshot 80c18f80-1949-4617-b561-695e43820ce1 (state Requested)...
362026-01-22T05:18:19.142ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-5b8d09a6-1708-4385-8ad0-c80611419d6e-80c18f80-1949-4617-b561-695e43820ce1
372026-01-22T05:18:19.146ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-5b8d09a6-1708-4385-8ad0-c80611419d6e-80c18f80-1949-4617-b561-695e43820ce1
382026-01-22T05:18:19.159ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-5b8d09a6-1708-4385-8ad0-c80611419d6e-80c18f80-1949-4617-b561-695e43820ce1
392026-01-22T05:18:19.160ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e/.zfs/snapshot/80c18f80-1949-4617-b561-695e43820ce1
402026-01-22T05:18:19.160ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-01-22T05:18:19.160ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-01-22T05:18:19.160ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432026-01-22T05:18:19.160ZINFOcrucible-agent (worker): commit
442026-01-22T05:18:19.164ZINFOcrucible-agent (worker): ok!
452026-01-22T05:18:19.168ZINFOcrucible-agent (worker): SMF ok!
462026-01-22T05:18:19.168ZINFOcrucible-agent (datafile): region 5b8d09a6-1708-4385-8ad0-c80611419d6e running snapshot 80c18f80-1949-4617-b561-695e43820ce1 state: Requested -> Created
472026-01-22T05:18:19.506ZINFOcrucible-agent (datafile): path is "/data/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e"
482026-01-22T05:18:19.512ZINFOcrucible-agent (datafile): dataset is oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e
492026-01-22T05:18:19.519ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e@80c18f80-1949-4617-b561-695e43820ce1
502026-01-22T05:18:19.519ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_34eb2046-018a-4339-8b3c-939760f8182f/crucible/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e", "80c18f80-1949-4617-b561-695e43820ce1"]
512026-01-22T05:18:19.525ZINFOcrucible-agent (dropshot): request completed latency_us = 19120 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:57018 req_id = 17d444f6-941e-4dcb-a912-c3588fef3e4f response_code = 200 uri = /crucible/0/regions/5b8d09a6-1708-4385-8ad0-c80611419d6e/snapshots/80c18f80-1949-4617-b561-695e43820ce1/run