Max level shown:
12026-04-10T21:02:52.762ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::c]:45431
22026-04-10T21:02:52.763ZINFOcrucible-agent (datafile): region d4f31d55-1c25-4262-bbff-102ff312e9ef state: Requested
32026-04-10T21:02:52.763ZINFOcrucible-agent (dropshot): request completed latency_us = 689 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:45431 req_id = b4a3012f-0b61-4358-bbfd-b850483a26d3 response_code = 200 uri = /crucible/0/regions
42026-04-10T21:02:52.763ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-04-10T21:02:52.771ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/d4f31d55-1c25-4262-bbff-102ff312e9ef
62026-04-10T21:02:52.771ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/d4f31d55-1c25-4262-bbff-102ff312e9ef
72026-04-10T21:02:52.923ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d4f31d55-1c25-4262-bbff-102ff312e9ef"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d4f31d55-1c25-4262-bbff-102ff312e9ef" region = d4f31d55-1c25-4262-bbff-102ff312e9ef
82026-04-10T21:02:52.923ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d4f31d55-1c25-4262-bbff-102ff312e9ef" "--data" "/data/regions/d4f31d55-1c25-4262-bbff-102ff312e9ef" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = d4f31d55-1c25-4262-bbff-102ff312e9ef
92026-04-10T21:02:53.059ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:45431 req_id = c7097472-e67d-444d-9703-a50e253c6bf5 response_code = 200 uri = /crucible/0/regions
102026-04-10T21:02:53.330ZINFOcrucible-agent (worker): region files created ok region = d4f31d55-1c25-4262-bbff-102ff312e9ef
112026-04-10T21:02:53.330ZINFOcrucible-agent (datafile): region d4f31d55-1c25-4262-bbff-102ff312e9ef state: Requested -> Created
122026-04-10T21:02:53.330ZINFOcrucible-agent (worker): applying SMF actions post create...
132026-04-10T21:02:53.349ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d4f31d55-1c25-4262-bbff-102ff312e9ef
142026-04-10T21:02:53.381ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d4f31d55-1c25-4262-bbff-102ff312e9ef
152026-04-10T21:02:53.434ZINFOcrucible-agent (worker): creating config property group
162026-04-10T21:02:53.471ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d4f31d55-1c25-4262-bbff-102ff312e9ef
172026-04-10T21:02:53.472ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d4f31d55-1c25-4262-bbff-102ff312e9ef
182026-04-10T21:02:53.472ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192026-04-10T21:02:53.472ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
202026-04-10T21:02:53.472ZINFOcrucible-agent (worker): commit
212026-04-10T21:02:53.484ZINFOcrucible-agent (worker): ok!
222026-04-10T21:02:53.524ZINFOcrucible-agent (worker): SMF ok!
232026-04-10T21:02:53.755ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:45431 req_id = 3c6c7dc7-0255-4dc8-b0aa-d16a206a5307 response_code = 200 uri = /crucible/0/regions
242026-04-10T21:02:55.432ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:45431 req_id = 0c0d99f4-14bd-410f-b8e3-64f9eff7a0d2 response_code = 200 uri = /crucible/0/regions/231def02-9951-42dd-9282-745ad609719a
252026-04-10T21:02:55.438ZINFOcrucible-agent (datafile): path is "/data/regions/231def02-9951-42dd-9282-745ad609719a"
262026-04-10T21:02:55.445ZINFOcrucible-agent (datafile): dataset is oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/231def02-9951-42dd-9282-745ad609719a
272026-04-10T21:02:55.452ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/231def02-9951-42dd-9282-745ad609719a@1043940e-9620-4324-be05-b16a319c6bf1
282026-04-10T21:02:55.452ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/231def02-9951-42dd-9282-745ad609719a", "1043940e-9620-4324-be05-b16a319c6bf1"]
292026-04-10T21:02:55.459ZINFOcrucible-agent (dropshot): request completed latency_us = 21177 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:45431 req_id = c7364e77-9d83-46b1-8cc0-5f14db89f3dc response_code = 200 uri = /crucible/0/regions/231def02-9951-42dd-9282-745ad609719a/snapshots/1043940e-9620-4324-be05-b16a319c6bf1
302026-04-10T21:02:55.472ZINFOcrucible-agent (datafile): path is "/data/regions/231def02-9951-42dd-9282-745ad609719a"
312026-04-10T21:02:55.479ZINFOcrucible-agent (datafile): dataset is oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/231def02-9951-42dd-9282-745ad609719a
322026-04-10T21:02:55.486ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/231def02-9951-42dd-9282-745ad609719a@1043940e-9620-4324-be05-b16a319c6bf1
332026-04-10T21:02:55.486ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/231def02-9951-42dd-9282-745ad609719a", "1043940e-9620-4324-be05-b16a319c6bf1"]
342026-04-10T21:02:55.498ZINFOcrucible-agent (datafile): requesting running snapshot 231def02-9951-42dd-9282-745ad609719a-1043940e-9620-4324-be05-b16a319c6bf1 state: Requested
352026-04-10T21:02:55.499ZINFOcrucible-agent (dropshot): request completed latency_us = 26136 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:45431 req_id = 7d99388d-10d8-46d9-a2ec-dc498ad510a9 response_code = 200 uri = /crucible/0/regions/231def02-9951-42dd-9282-745ad609719a/snapshots/1043940e-9620-4324-be05-b16a319c6bf1/run
362026-04-10T21:02:55.499ZINFOcrucible-agent (worker): applying SMF actions for region 231def02-9951-42dd-9282-745ad609719a running snapshot 1043940e-9620-4324-be05-b16a319c6bf1 (state Requested)...
372026-04-10T21:02:55.503ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-231def02-9951-42dd-9282-745ad609719a-1043940e-9620-4324-be05-b16a319c6bf1
382026-04-10T21:02:55.515ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-231def02-9951-42dd-9282-745ad609719a-1043940e-9620-4324-be05-b16a319c6bf1
392026-04-10T21:02:55.574ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-231def02-9951-42dd-9282-745ad609719a-1043940e-9620-4324-be05-b16a319c6bf1
402026-04-10T21:02:55.574ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/231def02-9951-42dd-9282-745ad609719a/.zfs/snapshot/1043940e-9620-4324-be05-b16a319c6bf1
412026-04-10T21:02:55.574ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
422026-04-10T21:02:55.574ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
432026-04-10T21:02:55.574ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
442026-04-10T21:02:55.574ZINFOcrucible-agent (worker): commit
452026-04-10T21:02:55.588ZINFOcrucible-agent (worker): ok!
462026-04-10T21:02:55.612ZINFOcrucible-agent (worker): SMF ok!
472026-04-10T21:02:55.612ZINFOcrucible-agent (datafile): region 231def02-9951-42dd-9282-745ad609719a running snapshot 1043940e-9620-4324-be05-b16a319c6bf1 state: Requested -> Created
482026-04-10T21:02:55.739ZINFOcrucible-agent (datafile): path is "/data/regions/231def02-9951-42dd-9282-745ad609719a"
492026-04-10T21:02:55.745ZINFOcrucible-agent (datafile): dataset is oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/231def02-9951-42dd-9282-745ad609719a
502026-04-10T21:02:55.752ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/231def02-9951-42dd-9282-745ad609719a@1043940e-9620-4324-be05-b16a319c6bf1
512026-04-10T21:02:55.752ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e78b13d7-0ea8-46a5-9dc2-d0f2b5240606/crucible/regions/231def02-9951-42dd-9282-745ad609719a", "1043940e-9620-4324-be05-b16a319c6bf1"]
522026-04-10T21:02:55.757ZINFOcrucible-agent (dropshot): request completed latency_us = 17861 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:45431 req_id = 8ee99f85-0b92-45c2-88a7-8f59b7dfd2df response_code = 200 uri = /crucible/0/regions/231def02-9951-42dd-9282-745ad609719a/snapshots/1043940e-9620-4324-be05-b16a319c6bf1/run