Max level shown:
12026-03-05T18:02:25.863ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:62547
22026-03-05T18:02:25.863ZINFOcrucible-agent (datafile): region e436b6f1-3376-4067-95ab-5979a4fd06ed state: Requested
32026-03-05T18:02:25.863ZINFOcrucible-agent (dropshot): request completed latency_us = 621 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62547 req_id = cc9929d8-e637-4543-8de1-90fc11b86fb9 response_code = 200 uri = /crucible/0/regions
42026-03-05T18:02:25.863ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-03-05T18:02:25.877ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/e436b6f1-3376-4067-95ab-5979a4fd06ed
62026-03-05T18:02:25.877ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/e436b6f1-3376-4067-95ab-5979a4fd06ed
72026-03-05T18:02:25.973ZINFOcrucible-agent (worker): creating region Region { id: RegionId("e436b6f1-3376-4067-95ab-5979a4fd06ed"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/e436b6f1-3376-4067-95ab-5979a4fd06ed" region = e436b6f1-3376-4067-95ab-5979a4fd06ed
82026-03-05T18:02:25.973ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "e436b6f1-3376-4067-95ab-5979a4fd06ed" "--data" "/data/regions/e436b6f1-3376-4067-95ab-5979a4fd06ed" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = e436b6f1-3376-4067-95ab-5979a4fd06ed
92026-03-05T18:02:26.203ZINFOcrucible-agent (worker): region files created ok region = e436b6f1-3376-4067-95ab-5979a4fd06ed
102026-03-05T18:02:26.203ZINFOcrucible-agent (datafile): region e436b6f1-3376-4067-95ab-5979a4fd06ed state: Requested -> Created
112026-03-05T18:02:26.204ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-03-05T18:02:26.215ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62547 req_id = 58eb82ed-b184-4188-bea1-90e80aafce80 response_code = 200 uri = /crucible/0/regions
132026-03-05T18:02:26.223ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-e436b6f1-3376-4067-95ab-5979a4fd06ed
142026-03-05T18:02:26.244ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-e436b6f1-3376-4067-95ab-5979a4fd06ed
152026-03-05T18:02:26.275ZINFOcrucible-agent (worker): creating config property group
162026-03-05T18:02:26.282ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-e436b6f1-3376-4067-95ab-5979a4fd06ed
172026-03-05T18:02:26.282ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e436b6f1-3376-4067-95ab-5979a4fd06ed
182026-03-05T18:02:26.282ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192026-03-05T18:02:26.282ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
202026-03-05T18:02:26.282ZINFOcrucible-agent (worker): commit
212026-03-05T18:02:26.292ZINFOcrucible-agent (worker): ok!
222026-03-05T18:02:26.299ZINFOcrucible-agent (worker): SMF ok!
232026-03-05T18:02:27.381ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62547 req_id = bba83f3f-48dc-403b-9dfa-00340109ad05 response_code = 200 uri = /crucible/0/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69
242026-03-05T18:02:27.384ZINFOcrucible-agent (datafile): path is "/data/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69"
252026-03-05T18:02:27.391ZINFOcrucible-agent (datafile): dataset is oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69
262026-03-05T18:02:27.397ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69@943e7588-f544-420d-994f-04a46104d1ac
272026-03-05T18:02:27.397ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69", "943e7588-f544-420d-994f-04a46104d1ac"]
282026-03-05T18:02:27.406ZINFOcrucible-agent (dropshot): request completed latency_us = 21257 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62547 req_id = f28d8fff-4878-4100-8241-46c06df217b5 response_code = 200 uri = /crucible/0/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69/snapshots/943e7588-f544-420d-994f-04a46104d1ac
292026-03-05T18:02:27.409ZINFOcrucible-agent (datafile): path is "/data/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69"
302026-03-05T18:02:27.415ZINFOcrucible-agent (datafile): dataset is oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69
312026-03-05T18:02:27.421ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69@943e7588-f544-420d-994f-04a46104d1ac
322026-03-05T18:02:27.421ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69", "943e7588-f544-420d-994f-04a46104d1ac"]
332026-03-05T18:02:27.430ZINFOcrucible-agent (datafile): requesting running snapshot 1b8d9d99-c44c-4049-a258-955c2f00dc69-943e7588-f544-420d-994f-04a46104d1ac state: Requested
342026-03-05T18:02:27.430ZINFOcrucible-agent (dropshot): request completed latency_us = 20479 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62547 req_id = 4e699dda-0b72-4387-b7f0-0567c9eb6781 response_code = 200 uri = /crucible/0/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69/snapshots/943e7588-f544-420d-994f-04a46104d1ac/run
352026-03-05T18:02:27.430ZINFOcrucible-agent (worker): applying SMF actions for region 1b8d9d99-c44c-4049-a258-955c2f00dc69 running snapshot 943e7588-f544-420d-994f-04a46104d1ac (state Requested)...
362026-03-05T18:02:27.435ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-1b8d9d99-c44c-4049-a258-955c2f00dc69-943e7588-f544-420d-994f-04a46104d1ac
372026-03-05T18:02:27.452ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-1b8d9d99-c44c-4049-a258-955c2f00dc69-943e7588-f544-420d-994f-04a46104d1ac
382026-03-05T18:02:27.470ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-1b8d9d99-c44c-4049-a258-955c2f00dc69-943e7588-f544-420d-994f-04a46104d1ac
392026-03-05T18:02:27.470ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69/.zfs/snapshot/943e7588-f544-420d-994f-04a46104d1ac
402026-03-05T18:02:27.470ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-03-05T18:02:27.470ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-03-05T18:02:27.470ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
432026-03-05T18:02:27.470ZINFOcrucible-agent (worker): commit
442026-03-05T18:02:27.475ZINFOcrucible-agent (worker): ok!
452026-03-05T18:02:27.482ZINFOcrucible-agent (worker): SMF ok!
462026-03-05T18:02:27.482ZINFOcrucible-agent (datafile): region 1b8d9d99-c44c-4049-a258-955c2f00dc69 running snapshot 943e7588-f544-420d-994f-04a46104d1ac state: Requested -> Created
472026-03-05T18:02:27.668ZINFOcrucible-agent (datafile): path is "/data/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69"
482026-03-05T18:02:27.674ZINFOcrucible-agent (datafile): dataset is oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69
492026-03-05T18:02:27.681ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69@943e7588-f544-420d-994f-04a46104d1ac
502026-03-05T18:02:27.681ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1fb74e6c-7654-4598-9eff-3750bf54efed/crucible/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69", "943e7588-f544-420d-994f-04a46104d1ac"]
512026-03-05T18:02:27.687ZINFOcrucible-agent (dropshot): request completed latency_us = 18168 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62547 req_id = f924456b-25cb-43a5-b4da-106de493c170 response_code = 200 uri = /crucible/0/regions/1b8d9d99-c44c-4049-a258-955c2f00dc69/snapshots/943e7588-f544-420d-994f-04a46104d1ac/run