Max level shown:
12026-05-27T02:51:38.744ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::b]:47391
22026-05-27T02:51:38.744ZINFOcrucible-agent (datafile): region 36d2ffd5-586d-4ac0-9231-187c8b18d190 state: Requested
32026-05-27T02:51:38.745ZINFOcrucible-agent (dropshot): request completed latency_us = 810 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:47391 req_id = f97b9efb-10de-4643-b6b7-d3db15c47d78 response_code = 200 uri = /crucible/0/regions
42026-05-27T02:51:38.745ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-05-27T02:51:38.754ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/36d2ffd5-586d-4ac0-9231-187c8b18d190
62026-05-27T02:51:38.754ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/36d2ffd5-586d-4ac0-9231-187c8b18d190
72026-05-27T02:51:38.792ZINFOcrucible-agent (worker): creating region Region { id: RegionId("36d2ffd5-586d-4ac0-9231-187c8b18d190"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/36d2ffd5-586d-4ac0-9231-187c8b18d190" region = 36d2ffd5-586d-4ac0-9231-187c8b18d190
82026-05-27T02:51:38.792ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "36d2ffd5-586d-4ac0-9231-187c8b18d190" "--data" "/data/regions/36d2ffd5-586d-4ac0-9231-187c8b18d190" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 36d2ffd5-586d-4ac0-9231-187c8b18d190
92026-05-27T02:51:38.844ZINFOcrucible-agent (worker): region files created ok region = 36d2ffd5-586d-4ac0-9231-187c8b18d190
102026-05-27T02:51:38.844ZINFOcrucible-agent (datafile): region 36d2ffd5-586d-4ac0-9231-187c8b18d190 state: Requested -> Created
112026-05-27T02:51:38.844ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-05-27T02:51:38.845ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-36d2ffd5-586d-4ac0-9231-187c8b18d190
132026-05-27T02:51:38.854ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-36d2ffd5-586d-4ac0-9231-187c8b18d190
142026-05-27T02:51:38.864ZINFOcrucible-agent (worker): creating config property group
152026-05-27T02:51:38.866ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-36d2ffd5-586d-4ac0-9231-187c8b18d190
162026-05-27T02:51:38.866ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/36d2ffd5-586d-4ac0-9231-187c8b18d190
172026-05-27T02:51:38.866ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-05-27T02:51:38.866ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
192026-05-27T02:51:38.866ZINFOcrucible-agent (worker): commit
202026-05-27T02:51:38.868ZINFOcrucible-agent (worker): ok!
212026-05-27T02:51:38.871ZINFOcrucible-agent (worker): SMF ok!
222026-05-27T02:51:38.925ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:47391 req_id = e40f8b26-1650-4a5f-9726-bc05cc7f6c59 response_code = 200 uri = /crucible/0/regions
232026-05-27T02:51:40.851ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:47391 req_id = 7507661d-295e-4c9e-822d-3c27ed6b6382 response_code = 200 uri = /crucible/0/regions/b4688725-0163-4e18-9c9a-7ddf054cc805
242026-05-27T02:51:40.854ZINFOcrucible-agent (datafile): path is "/data/regions/b4688725-0163-4e18-9c9a-7ddf054cc805"
252026-05-27T02:51:40.861ZINFOcrucible-agent (datafile): dataset is oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/b4688725-0163-4e18-9c9a-7ddf054cc805
262026-05-27T02:51:40.867ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/b4688725-0163-4e18-9c9a-7ddf054cc805@ad7d7433-7d37-4ace-96d2-2035442e7e77
272026-05-27T02:51:40.867ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/b4688725-0163-4e18-9c9a-7ddf054cc805", "ad7d7433-7d37-4ace-96d2-2035442e7e77"]
282026-05-27T02:51:40.874ZINFOcrucible-agent (dropshot): request completed latency_us = 20115 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:47391 req_id = 02bf0b13-2afa-4bcf-85a2-68e9e7095a83 response_code = 200 uri = /crucible/0/regions/b4688725-0163-4e18-9c9a-7ddf054cc805/snapshots/ad7d7433-7d37-4ace-96d2-2035442e7e77
292026-05-27T02:51:40.879ZINFOcrucible-agent (datafile): path is "/data/regions/b4688725-0163-4e18-9c9a-7ddf054cc805"
302026-05-27T02:51:40.886ZINFOcrucible-agent (datafile): dataset is oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/b4688725-0163-4e18-9c9a-7ddf054cc805
312026-05-27T02:51:40.891ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/b4688725-0163-4e18-9c9a-7ddf054cc805@ad7d7433-7d37-4ace-96d2-2035442e7e77
322026-05-27T02:51:40.891ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/b4688725-0163-4e18-9c9a-7ddf054cc805", "ad7d7433-7d37-4ace-96d2-2035442e7e77"]
332026-05-27T02:51:40.897ZINFOcrucible-agent (datafile): requesting running snapshot b4688725-0163-4e18-9c9a-7ddf054cc805-ad7d7433-7d37-4ace-96d2-2035442e7e77 state: Requested
342026-05-27T02:51:40.897ZINFOcrucible-agent (worker): applying SMF actions for region b4688725-0163-4e18-9c9a-7ddf054cc805 running snapshot ad7d7433-7d37-4ace-96d2-2035442e7e77 (state Requested)...
352026-05-27T02:51:40.897ZINFOcrucible-agent (dropshot): request completed latency_us = 18620 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:47391 req_id = 22d18602-c081-4f5b-a22d-16890a9b7f35 response_code = 200 uri = /crucible/0/regions/b4688725-0163-4e18-9c9a-7ddf054cc805/snapshots/ad7d7433-7d37-4ace-96d2-2035442e7e77/run
362026-05-27T02:51:40.902ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-b4688725-0163-4e18-9c9a-7ddf054cc805-ad7d7433-7d37-4ace-96d2-2035442e7e77
372026-05-27T02:51:40.906ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-b4688725-0163-4e18-9c9a-7ddf054cc805-ad7d7433-7d37-4ace-96d2-2035442e7e77
382026-05-27T02:51:40.912ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-b4688725-0163-4e18-9c9a-7ddf054cc805-ad7d7433-7d37-4ace-96d2-2035442e7e77
392026-05-27T02:51:40.912ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b4688725-0163-4e18-9c9a-7ddf054cc805/.zfs/snapshot/ad7d7433-7d37-4ace-96d2-2035442e7e77
402026-05-27T02:51:40.912ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-05-27T02:51:40.912ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-05-27T02:51:40.912ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
432026-05-27T02:51:40.912ZINFOcrucible-agent (worker): commit
442026-05-27T02:51:40.914ZINFOcrucible-agent (worker): ok!
452026-05-27T02:51:40.917ZINFOcrucible-agent (worker): SMF ok!
462026-05-27T02:51:40.917ZINFOcrucible-agent (datafile): region b4688725-0163-4e18-9c9a-7ddf054cc805 running snapshot ad7d7433-7d37-4ace-96d2-2035442e7e77 state: Requested -> Created
472026-05-27T02:51:41.210ZINFOcrucible-agent (datafile): path is "/data/regions/b4688725-0163-4e18-9c9a-7ddf054cc805"
482026-05-27T02:51:41.217ZINFOcrucible-agent (datafile): dataset is oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/b4688725-0163-4e18-9c9a-7ddf054cc805
492026-05-27T02:51:41.225ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/b4688725-0163-4e18-9c9a-7ddf054cc805@ad7d7433-7d37-4ace-96d2-2035442e7e77
502026-05-27T02:51:41.225ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_936fc315-a21b-480b-bab2-7d768ff75e78/crucible/regions/b4688725-0163-4e18-9c9a-7ddf054cc805", "ad7d7433-7d37-4ace-96d2-2035442e7e77"]
512026-05-27T02:51:41.231ZINFOcrucible-agent (dropshot): request completed latency_us = 21000 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:47391 req_id = b0573094-fa32-40ae-888d-8ca6c22a825b response_code = 200 uri = /crucible/0/regions/b4688725-0163-4e18-9c9a-7ddf054cc805/snapshots/ad7d7433-7d37-4ace-96d2-2035442e7e77/run