Max level shown:
12026-03-05T03:46:30.736ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:35187
22026-03-05T03:46:30.736ZINFOcrucible-agent (datafile): region c224329d-d1e7-40cb-ace7-4d1698d33958 state: Requested
32026-03-05T03:46:30.737ZINFOcrucible-agent (dropshot): request completed latency_us = 701 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35187 req_id = 280f9bb2-2c52-4e23-8f0c-4a41c55de9e8 response_code = 200 uri = /crucible/0/regions
42026-03-05T03:46:30.737ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-03-05T03:46:30.746ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/c224329d-d1e7-40cb-ace7-4d1698d33958
62026-03-05T03:46:30.746ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/c224329d-d1e7-40cb-ace7-4d1698d33958
72026-03-05T03:46:30.799ZINFOcrucible-agent (worker): creating region Region { id: RegionId("c224329d-d1e7-40cb-ace7-4d1698d33958"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/c224329d-d1e7-40cb-ace7-4d1698d33958" region = c224329d-d1e7-40cb-ace7-4d1698d33958
82026-03-05T03:46:30.799ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "c224329d-d1e7-40cb-ace7-4d1698d33958" "--data" "/data/regions/c224329d-d1e7-40cb-ace7-4d1698d33958" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = c224329d-d1e7-40cb-ace7-4d1698d33958
92026-03-05T03:46:30.895ZINFOcrucible-agent (worker): region files created ok region = c224329d-d1e7-40cb-ace7-4d1698d33958
102026-03-05T03:46:30.895ZINFOcrucible-agent (datafile): region c224329d-d1e7-40cb-ace7-4d1698d33958 state: Requested -> Created
112026-03-05T03:46:30.895ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-03-05T03:46:30.917ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-c224329d-d1e7-40cb-ace7-4d1698d33958
132026-03-05T03:46:30.923ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-c224329d-d1e7-40cb-ace7-4d1698d33958
142026-03-05T03:46:30.932ZINFOcrucible-agent (worker): creating config property group
152026-03-05T03:46:30.935ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-c224329d-d1e7-40cb-ace7-4d1698d33958
162026-03-05T03:46:30.935ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/c224329d-d1e7-40cb-ace7-4d1698d33958
172026-03-05T03:46:30.935ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-03-05T03:46:30.935ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
192026-03-05T03:46:30.935ZINFOcrucible-agent (worker): commit
202026-03-05T03:46:30.939ZINFOcrucible-agent (worker): ok!
212026-03-05T03:46:30.942ZINFOcrucible-agent (worker): SMF ok!
222026-03-05T03:46:31.105ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35187 req_id = d94725b7-84d5-4836-9330-8d33993f3b00 response_code = 200 uri = /crucible/0/regions
232026-03-05T03:46:32.994ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:35187 req_id = 6f64500b-2c7e-40e8-8039-a6748b4d233f response_code = 200 uri = /crucible/0/regions/36e28a32-ed2d-4103-a85b-0381af5fef96
242026-03-05T03:46:32.997ZINFOcrucible-agent (datafile): path is "/data/regions/36e28a32-ed2d-4103-a85b-0381af5fef96"
252026-03-05T03:46:33.004ZINFOcrucible-agent (datafile): dataset is oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/36e28a32-ed2d-4103-a85b-0381af5fef96
262026-03-05T03:46:33.011ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/36e28a32-ed2d-4103-a85b-0381af5fef96@7096ab6d-0ec0-44c8-be67-8f0b18695dba
272026-03-05T03:46:33.011ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/36e28a32-ed2d-4103-a85b-0381af5fef96", "7096ab6d-0ec0-44c8-be67-8f0b18695dba"]
282026-03-05T03:46:33.019ZINFOcrucible-agent (dropshot): request completed latency_us = 22687 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:35187 req_id = 6fbe78be-9845-44c0-8a53-c7981a8be161 response_code = 200 uri = /crucible/0/regions/36e28a32-ed2d-4103-a85b-0381af5fef96/snapshots/7096ab6d-0ec0-44c8-be67-8f0b18695dba
292026-03-05T03:46:33.023ZINFOcrucible-agent (datafile): path is "/data/regions/36e28a32-ed2d-4103-a85b-0381af5fef96"
302026-03-05T03:46:33.030ZINFOcrucible-agent (datafile): dataset is oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/36e28a32-ed2d-4103-a85b-0381af5fef96
312026-03-05T03:46:33.037ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/36e28a32-ed2d-4103-a85b-0381af5fef96@7096ab6d-0ec0-44c8-be67-8f0b18695dba
322026-03-05T03:46:33.037ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/36e28a32-ed2d-4103-a85b-0381af5fef96", "7096ab6d-0ec0-44c8-be67-8f0b18695dba"]
332026-03-05T03:46:33.045ZINFOcrucible-agent (datafile): requesting running snapshot 36e28a32-ed2d-4103-a85b-0381af5fef96-7096ab6d-0ec0-44c8-be67-8f0b18695dba state: Requested
342026-03-05T03:46:33.045ZINFOcrucible-agent (dropshot): request completed latency_us = 21961 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35187 req_id = 8eaa211f-57eb-4354-932e-ecceff53251e response_code = 200 uri = /crucible/0/regions/36e28a32-ed2d-4103-a85b-0381af5fef96/snapshots/7096ab6d-0ec0-44c8-be67-8f0b18695dba/run
352026-03-05T03:46:33.045ZINFOcrucible-agent (worker): applying SMF actions for region 36e28a32-ed2d-4103-a85b-0381af5fef96 running snapshot 7096ab6d-0ec0-44c8-be67-8f0b18695dba (state Requested)...
362026-03-05T03:46:33.049ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-36e28a32-ed2d-4103-a85b-0381af5fef96-7096ab6d-0ec0-44c8-be67-8f0b18695dba
372026-03-05T03:46:33.063ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-36e28a32-ed2d-4103-a85b-0381af5fef96-7096ab6d-0ec0-44c8-be67-8f0b18695dba
382026-03-05T03:46:33.075ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-36e28a32-ed2d-4103-a85b-0381af5fef96-7096ab6d-0ec0-44c8-be67-8f0b18695dba
392026-03-05T03:46:33.075ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/36e28a32-ed2d-4103-a85b-0381af5fef96/.zfs/snapshot/7096ab6d-0ec0-44c8-be67-8f0b18695dba
402026-03-05T03:46:33.075ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-03-05T03:46:33.075ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-03-05T03:46:33.075ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
432026-03-05T03:46:33.075ZINFOcrucible-agent (worker): commit
442026-03-05T03:46:33.079ZINFOcrucible-agent (worker): ok!
452026-03-05T03:46:33.082ZINFOcrucible-agent (worker): SMF ok!
462026-03-05T03:46:33.082ZINFOcrucible-agent (datafile): region 36e28a32-ed2d-4103-a85b-0381af5fef96 running snapshot 7096ab6d-0ec0-44c8-be67-8f0b18695dba state: Requested -> Created
472026-03-05T03:46:33.231ZINFOcrucible-agent (datafile): path is "/data/regions/36e28a32-ed2d-4103-a85b-0381af5fef96"
482026-03-05T03:46:33.237ZINFOcrucible-agent (datafile): dataset is oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/36e28a32-ed2d-4103-a85b-0381af5fef96
492026-03-05T03:46:33.244ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/36e28a32-ed2d-4103-a85b-0381af5fef96@7096ab6d-0ec0-44c8-be67-8f0b18695dba
502026-03-05T03:46:33.244ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b897729e-e5d9-4343-9fd0-dc41055ef9df/crucible/regions/36e28a32-ed2d-4103-a85b-0381af5fef96", "7096ab6d-0ec0-44c8-be67-8f0b18695dba"]
512026-03-05T03:46:33.250ZINFOcrucible-agent (dropshot): request completed latency_us = 19459 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:35187 req_id = 44d0f8b6-e18d-4c6e-8ed9-1c2b48237976 response_code = 200 uri = /crucible/0/regions/36e28a32-ed2d-4103-a85b-0381af5fef96/snapshots/7096ab6d-0ec0-44c8-be67-8f0b18695dba/run