Max level shown:
12026-03-04T19:05:54.816ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:44838
22026-03-04T19:05:54.816ZINFOcrucible-agent (datafile): region c3869c75-29e4-4a38-9752-673eabe04772 state: Requested
32026-03-04T19:05:54.817ZINFOcrucible-agent (dropshot): request completed latency_us = 979 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44838 req_id = 27ca6b65-dd4a-4f5c-9244-4c87f91a1c87 response_code = 200 uri = /crucible/0/regions
42026-03-04T19:05:54.817ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-03-04T19:05:54.825ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/c3869c75-29e4-4a38-9752-673eabe04772
62026-03-04T19:05:54.825ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/c3869c75-29e4-4a38-9752-673eabe04772
72026-03-04T19:05:54.890ZINFOcrucible-agent (worker): creating region Region { id: RegionId("c3869c75-29e4-4a38-9752-673eabe04772"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/c3869c75-29e4-4a38-9752-673eabe04772" region = c3869c75-29e4-4a38-9752-673eabe04772
82026-03-04T19:05:54.890ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "c3869c75-29e4-4a38-9752-673eabe04772" "--data" "/data/regions/c3869c75-29e4-4a38-9752-673eabe04772" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = c3869c75-29e4-4a38-9752-673eabe04772
92026-03-04T19:05:55.069ZINFOcrucible-agent (worker): region files created ok region = c3869c75-29e4-4a38-9752-673eabe04772
102026-03-04T19:05:55.069ZINFOcrucible-agent (datafile): region c3869c75-29e4-4a38-9752-673eabe04772 state: Requested -> Created
112026-03-04T19:05:55.069ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-03-04T19:05:55.096ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-c3869c75-29e4-4a38-9752-673eabe04772
132026-03-04T19:05:55.117ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-c3869c75-29e4-4a38-9752-673eabe04772
142026-03-04T19:05:55.137ZINFOcrucible-agent (worker): creating config property group
152026-03-04T19:05:55.144ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-c3869c75-29e4-4a38-9752-673eabe04772
162026-03-04T19:05:55.145ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/c3869c75-29e4-4a38-9752-673eabe04772
172026-03-04T19:05:55.145ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-03-04T19:05:55.145ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
192026-03-04T19:05:55.145ZINFOcrucible-agent (worker): commit
202026-03-04T19:05:55.150ZINFOcrucible-agent (worker): ok!
212026-03-04T19:05:55.159ZINFOcrucible-agent (worker): SMF ok!
222026-03-04T19:05:55.163ZINFOcrucible-agent (dropshot): request completed latency_us = 93 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44838 req_id = a04d7166-85e1-495f-a2df-760d89ac084b response_code = 200 uri = /crucible/0/regions
232026-03-04T19:05:55.955ZINFOcrucible-agent (dropshot): request completed latency_us = 51 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:44838 req_id = ee93ef30-bfdd-4cb7-8a2d-cda235750d3d response_code = 200 uri = /crucible/0/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9
242026-03-04T19:05:55.958ZINFOcrucible-agent (datafile): path is "/data/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9"
252026-03-04T19:05:55.965ZINFOcrucible-agent (datafile): dataset is oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9
262026-03-04T19:05:55.972ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9@4c7377e1-6fa0-42de-9f51-777609c25af2
272026-03-04T19:05:55.972ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9", "4c7377e1-6fa0-42de-9f51-777609c25af2"]
282026-03-04T19:05:55.980ZINFOcrucible-agent (dropshot): request completed latency_us = 22049 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:44838 req_id = 82ec535c-4432-4721-9c85-c70b14b67403 response_code = 200 uri = /crucible/0/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9/snapshots/4c7377e1-6fa0-42de-9f51-777609c25af2
292026-03-04T19:05:55.984ZINFOcrucible-agent (datafile): path is "/data/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9"
302026-03-04T19:05:55.991ZINFOcrucible-agent (datafile): dataset is oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9
312026-03-04T19:05:55.997ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9@4c7377e1-6fa0-42de-9f51-777609c25af2
322026-03-04T19:05:55.997ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9", "4c7377e1-6fa0-42de-9f51-777609c25af2"]
332026-03-04T19:05:56.005ZINFOcrucible-agent (datafile): requesting running snapshot a14b6bbb-f0c8-4c14-9caa-777bf401eae9-4c7377e1-6fa0-42de-9f51-777609c25af2 state: Requested
342026-03-04T19:05:56.005ZINFOcrucible-agent (dropshot): request completed latency_us = 21627 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44838 req_id = 06ed78b2-94ef-4f17-a91a-14abfb77729d response_code = 200 uri = /crucible/0/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9/snapshots/4c7377e1-6fa0-42de-9f51-777609c25af2/run
352026-03-04T19:05:56.005ZINFOcrucible-agent (worker): applying SMF actions for region a14b6bbb-f0c8-4c14-9caa-777bf401eae9 running snapshot 4c7377e1-6fa0-42de-9f51-777609c25af2 (state Requested)...
362026-03-04T19:05:56.010ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-a14b6bbb-f0c8-4c14-9caa-777bf401eae9-4c7377e1-6fa0-42de-9f51-777609c25af2
372026-03-04T19:05:56.018ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-a14b6bbb-f0c8-4c14-9caa-777bf401eae9-4c7377e1-6fa0-42de-9f51-777609c25af2
382026-03-04T19:05:56.028ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-a14b6bbb-f0c8-4c14-9caa-777bf401eae9-4c7377e1-6fa0-42de-9f51-777609c25af2
392026-03-04T19:05:56.028ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9/.zfs/snapshot/4c7377e1-6fa0-42de-9f51-777609c25af2
402026-03-04T19:05:56.028ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-03-04T19:05:56.028ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-03-04T19:05:56.028ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
432026-03-04T19:05:56.028ZINFOcrucible-agent (worker): commit
442026-03-04T19:05:56.032ZINFOcrucible-agent (worker): ok!
452026-03-04T19:05:56.037ZINFOcrucible-agent (worker): SMF ok!
462026-03-04T19:05:56.037ZINFOcrucible-agent (datafile): region a14b6bbb-f0c8-4c14-9caa-777bf401eae9 running snapshot 4c7377e1-6fa0-42de-9f51-777609c25af2 state: Requested -> Created
472026-03-04T19:05:56.377ZINFOcrucible-agent (datafile): path is "/data/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9"
482026-03-04T19:05:56.384ZINFOcrucible-agent (datafile): dataset is oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9
492026-03-04T19:05:56.389ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9@4c7377e1-6fa0-42de-9f51-777609c25af2
502026-03-04T19:05:56.389ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b3720e1a-a843-474c-8872-2ae61246642e/crucible/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9", "4c7377e1-6fa0-42de-9f51-777609c25af2"]
512026-03-04T19:05:56.396ZINFOcrucible-agent (dropshot): request completed latency_us = 19079 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44838 req_id = 8be660d7-5328-45c9-a8a4-b2040b526b73 response_code = 200 uri = /crucible/0/regions/a14b6bbb-f0c8-4c14-9caa-777bf401eae9/snapshots/4c7377e1-6fa0-42de-9f51-777609c25af2/run