Max level shown:
12026-02-25T21:53:17.053ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:58735
22026-02-25T21:53:17.053ZINFOcrucible-agent (datafile): region 50e006b8-3546-4f5b-b158-72689208b940 state: Requested
32026-02-25T21:53:17.054ZINFOcrucible-agent (dropshot): request completed latency_us = 970 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:58735 req_id = 314b5b4b-9c7a-4b55-9054-5029c29aa365 response_code = 200 uri = /crucible/0/regions
42026-02-25T21:53:17.054ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-02-25T21:53:17.061ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/50e006b8-3546-4f5b-b158-72689208b940
62026-02-25T21:53:17.061ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/50e006b8-3546-4f5b-b158-72689208b940
72026-02-25T21:53:17.110ZINFOcrucible-agent (worker): creating region Region { id: RegionId("50e006b8-3546-4f5b-b158-72689208b940"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/50e006b8-3546-4f5b-b158-72689208b940" region = 50e006b8-3546-4f5b-b158-72689208b940
82026-02-25T21:53:17.110ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "50e006b8-3546-4f5b-b158-72689208b940" "--data" "/data/regions/50e006b8-3546-4f5b-b158-72689208b940" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 50e006b8-3546-4f5b-b158-72689208b940
92026-02-25T21:53:17.190ZINFOcrucible-agent (worker): region files created ok region = 50e006b8-3546-4f5b-b158-72689208b940
102026-02-25T21:53:17.190ZINFOcrucible-agent (datafile): region 50e006b8-3546-4f5b-b158-72689208b940 state: Requested -> Created
112026-02-25T21:53:17.190ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-02-25T21:53:17.191ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-50e006b8-3546-4f5b-b158-72689208b940
132026-02-25T21:53:17.204ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-50e006b8-3546-4f5b-b158-72689208b940
142026-02-25T21:53:17.227ZINFOcrucible-agent (worker): creating config property group
152026-02-25T21:53:17.230ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-50e006b8-3546-4f5b-b158-72689208b940
162026-02-25T21:53:17.230ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/50e006b8-3546-4f5b-b158-72689208b940
172026-02-25T21:53:17.231ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-02-25T21:53:17.231ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192026-02-25T21:53:17.231ZINFOcrucible-agent (worker): commit
202026-02-25T21:53:17.234ZINFOcrucible-agent (worker): ok!
212026-02-25T21:53:17.248ZINFOcrucible-agent (worker): SMF ok!
222026-02-25T21:53:17.301ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:58735 req_id = 2c1499c3-1f32-4f86-9972-3f2fa3604524 response_code = 200 uri = /crucible/0/regions
232026-02-25T21:53:18.824ZINFOcrucible-agent (dropshot): request completed latency_us = 51 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:58735 req_id = 95f5bc7b-c9b0-468e-941c-3fe3fecb8629 response_code = 200 uri = /crucible/0/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d
242026-02-25T21:53:18.827ZINFOcrucible-agent (datafile): path is "/data/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d"
252026-02-25T21:53:18.834ZINFOcrucible-agent (datafile): dataset is oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d
262026-02-25T21:53:18.840ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d@bc4f5f82-ab72-4f9a-bb10-16f243bc8814
272026-02-25T21:53:18.840ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d", "bc4f5f82-ab72-4f9a-bb10-16f243bc8814"]
282026-02-25T21:53:18.848ZINFOcrucible-agent (dropshot): request completed latency_us = 20588 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:58735 req_id = b8e7372d-752e-4968-aac3-e110ea6ef6cc response_code = 200 uri = /crucible/0/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d/snapshots/bc4f5f82-ab72-4f9a-bb10-16f243bc8814
292026-02-25T21:53:18.852ZINFOcrucible-agent (datafile): path is "/data/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d"
302026-02-25T21:53:18.858ZINFOcrucible-agent (datafile): dataset is oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d
312026-02-25T21:53:18.865ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d@bc4f5f82-ab72-4f9a-bb10-16f243bc8814
322026-02-25T21:53:18.865ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d", "bc4f5f82-ab72-4f9a-bb10-16f243bc8814"]
332026-02-25T21:53:18.871ZINFOcrucible-agent (datafile): requesting running snapshot 83ce9665-3fbc-409e-a327-37434ab0dc2d-bc4f5f82-ab72-4f9a-bb10-16f243bc8814 state: Requested
342026-02-25T21:53:18.872ZINFOcrucible-agent (dropshot): request completed latency_us = 19815 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:58735 req_id = 010ccdd3-8dde-4d5d-9bd2-907eab18e40c response_code = 200 uri = /crucible/0/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d/snapshots/bc4f5f82-ab72-4f9a-bb10-16f243bc8814/run
352026-02-25T21:53:18.872ZINFOcrucible-agent (worker): applying SMF actions for region 83ce9665-3fbc-409e-a327-37434ab0dc2d running snapshot bc4f5f82-ab72-4f9a-bb10-16f243bc8814 (state Requested)...
362026-02-25T21:53:18.877ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-83ce9665-3fbc-409e-a327-37434ab0dc2d-bc4f5f82-ab72-4f9a-bb10-16f243bc8814
372026-02-25T21:53:18.885ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-83ce9665-3fbc-409e-a327-37434ab0dc2d-bc4f5f82-ab72-4f9a-bb10-16f243bc8814
382026-02-25T21:53:18.895ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-83ce9665-3fbc-409e-a327-37434ab0dc2d-bc4f5f82-ab72-4f9a-bb10-16f243bc8814
392026-02-25T21:53:18.895ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d/.zfs/snapshot/bc4f5f82-ab72-4f9a-bb10-16f243bc8814
402026-02-25T21:53:18.895ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-02-25T21:53:18.895ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-02-25T21:53:18.895ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432026-02-25T21:53:18.895ZINFOcrucible-agent (worker): commit
442026-02-25T21:53:18.899ZINFOcrucible-agent (worker): ok!
452026-02-25T21:53:18.902ZINFOcrucible-agent (worker): SMF ok!
462026-02-25T21:53:18.902ZINFOcrucible-agent (datafile): region 83ce9665-3fbc-409e-a327-37434ab0dc2d running snapshot bc4f5f82-ab72-4f9a-bb10-16f243bc8814 state: Requested -> Created
472026-02-25T21:53:19.200ZINFOcrucible-agent (datafile): path is "/data/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d"
482026-02-25T21:53:19.206ZINFOcrucible-agent (datafile): dataset is oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d
492026-02-25T21:53:19.212ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d@bc4f5f82-ab72-4f9a-bb10-16f243bc8814
502026-02-25T21:53:19.212ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_41ce5bad-939a-427e-817f-98a572e212b9/crucible/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d", "bc4f5f82-ab72-4f9a-bb10-16f243bc8814"]
512026-02-25T21:53:19.218ZINFOcrucible-agent (dropshot): request completed latency_us = 18612 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:58735 req_id = 9314d5ac-693f-48ce-9476-43491fa1d0ab response_code = 200 uri = /crucible/0/regions/83ce9665-3fbc-409e-a327-37434ab0dc2d/snapshots/bc4f5f82-ab72-4f9a-bb10-16f243bc8814/run