Max level shown:
12026-03-24T18:10:12.110ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::b]:43632
22026-03-24T18:10:12.110ZINFOcrucible-agent (datafile): region f359bb90-b860-4a9b-bc86-581899f6c8ed state: Requested
32026-03-24T18:10:12.111ZINFOcrucible-agent (dropshot): request completed latency_us = 1301 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43632 req_id = 3a4f3746-591d-4f88-8518-2c769391e021 response_code = 200 uri = /crucible/0/regions
42026-03-24T18:10:12.111ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-03-24T18:10:12.121ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/f359bb90-b860-4a9b-bc86-581899f6c8ed
62026-03-24T18:10:12.121ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/f359bb90-b860-4a9b-bc86-581899f6c8ed
72026-03-24T18:10:12.214ZINFOcrucible-agent (worker): creating region Region { id: RegionId("f359bb90-b860-4a9b-bc86-581899f6c8ed"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/f359bb90-b860-4a9b-bc86-581899f6c8ed" region = f359bb90-b860-4a9b-bc86-581899f6c8ed
82026-03-24T18:10:12.214ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "f359bb90-b860-4a9b-bc86-581899f6c8ed" "--data" "/data/regions/f359bb90-b860-4a9b-bc86-581899f6c8ed" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = f359bb90-b860-4a9b-bc86-581899f6c8ed
92026-03-24T18:10:12.423ZINFOcrucible-agent (worker): region files created ok region = f359bb90-b860-4a9b-bc86-581899f6c8ed
102026-03-24T18:10:12.423ZINFOcrucible-agent (datafile): region f359bb90-b860-4a9b-bc86-581899f6c8ed state: Requested -> Created
112026-03-24T18:10:12.423ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-03-24T18:10:12.452ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-f359bb90-b860-4a9b-bc86-581899f6c8ed
132026-03-24T18:10:12.460ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43632 req_id = 2f96bcd3-3e70-4fb8-8a20-f46157f42b34 response_code = 200 uri = /crucible/0/regions
142026-03-24T18:10:12.482ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-f359bb90-b860-4a9b-bc86-581899f6c8ed
152026-03-24T18:10:12.510ZINFOcrucible-agent (worker): creating config property group
162026-03-24T18:10:12.521ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-f359bb90-b860-4a9b-bc86-581899f6c8ed
172026-03-24T18:10:12.522ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f359bb90-b860-4a9b-bc86-581899f6c8ed
182026-03-24T18:10:12.522ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192026-03-24T18:10:12.522ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
202026-03-24T18:10:12.522ZINFOcrucible-agent (worker): commit
212026-03-24T18:10:12.531ZINFOcrucible-agent (worker): ok!
222026-03-24T18:10:12.538ZINFOcrucible-agent (worker): SMF ok!
232026-03-24T18:10:13.823ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43632 req_id = 19bea55b-4664-4fed-ac6c-1bc4ed022e5a response_code = 200 uri = /crucible/0/regions/3f796b11-b103-4585-aee9-a1878b120734
242026-03-24T18:10:13.826ZINFOcrucible-agent (datafile): path is "/data/regions/3f796b11-b103-4585-aee9-a1878b120734"
252026-03-24T18:10:13.832ZINFOcrucible-agent (datafile): dataset is oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/3f796b11-b103-4585-aee9-a1878b120734
262026-03-24T18:10:13.837ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/3f796b11-b103-4585-aee9-a1878b120734@daf70307-7db2-49dc-ada4-0c7ea02b542f
272026-03-24T18:10:13.837ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/3f796b11-b103-4585-aee9-a1878b120734", "daf70307-7db2-49dc-ada4-0c7ea02b542f"]
282026-03-24T18:10:13.844ZINFOcrucible-agent (dropshot): request completed latency_us = 18346 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43632 req_id = 6e605168-8cf2-473b-af2e-c67e1d191b53 response_code = 200 uri = /crucible/0/regions/3f796b11-b103-4585-aee9-a1878b120734/snapshots/daf70307-7db2-49dc-ada4-0c7ea02b542f
292026-03-24T18:10:13.848ZINFOcrucible-agent (datafile): path is "/data/regions/3f796b11-b103-4585-aee9-a1878b120734"
302026-03-24T18:10:13.853ZINFOcrucible-agent (datafile): dataset is oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/3f796b11-b103-4585-aee9-a1878b120734
312026-03-24T18:10:13.858ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/3f796b11-b103-4585-aee9-a1878b120734@daf70307-7db2-49dc-ada4-0c7ea02b542f
322026-03-24T18:10:13.858ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/3f796b11-b103-4585-aee9-a1878b120734", "daf70307-7db2-49dc-ada4-0c7ea02b542f"]
332026-03-24T18:10:13.864ZINFOcrucible-agent (datafile): requesting running snapshot 3f796b11-b103-4585-aee9-a1878b120734-daf70307-7db2-49dc-ada4-0c7ea02b542f state: Requested
342026-03-24T18:10:13.865ZINFOcrucible-agent (dropshot): request completed latency_us = 16529 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43632 req_id = 9fc92535-3f7d-4352-9a89-4292be0edd04 response_code = 200 uri = /crucible/0/regions/3f796b11-b103-4585-aee9-a1878b120734/snapshots/daf70307-7db2-49dc-ada4-0c7ea02b542f/run
352026-03-24T18:10:13.865ZINFOcrucible-agent (worker): applying SMF actions for region 3f796b11-b103-4585-aee9-a1878b120734 running snapshot daf70307-7db2-49dc-ada4-0c7ea02b542f (state Requested)...
362026-03-24T18:10:13.869ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-3f796b11-b103-4585-aee9-a1878b120734-daf70307-7db2-49dc-ada4-0c7ea02b542f
372026-03-24T18:10:13.878ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-3f796b11-b103-4585-aee9-a1878b120734-daf70307-7db2-49dc-ada4-0c7ea02b542f
382026-03-24T18:10:13.909ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-3f796b11-b103-4585-aee9-a1878b120734-daf70307-7db2-49dc-ada4-0c7ea02b542f
392026-03-24T18:10:13.909ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3f796b11-b103-4585-aee9-a1878b120734/.zfs/snapshot/daf70307-7db2-49dc-ada4-0c7ea02b542f
402026-03-24T18:10:13.909ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-03-24T18:10:13.909ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-03-24T18:10:13.909ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432026-03-24T18:10:13.910ZINFOcrucible-agent (worker): commit
442026-03-24T18:10:13.916ZINFOcrucible-agent (worker): ok!
452026-03-24T18:10:13.929ZINFOcrucible-agent (worker): SMF ok!
462026-03-24T18:10:13.929ZINFOcrucible-agent (datafile): region 3f796b11-b103-4585-aee9-a1878b120734 running snapshot daf70307-7db2-49dc-ada4-0c7ea02b542f state: Requested -> Created
472026-03-24T18:10:14.116ZINFOcrucible-agent (datafile): path is "/data/regions/3f796b11-b103-4585-aee9-a1878b120734"
482026-03-24T18:10:14.123ZINFOcrucible-agent (datafile): dataset is oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/3f796b11-b103-4585-aee9-a1878b120734
492026-03-24T18:10:14.129ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/3f796b11-b103-4585-aee9-a1878b120734@daf70307-7db2-49dc-ada4-0c7ea02b542f
502026-03-24T18:10:14.129ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_bc3c3b47-071d-4f54-8ee5-ac57d25f4501/crucible/regions/3f796b11-b103-4585-aee9-a1878b120734", "daf70307-7db2-49dc-ada4-0c7ea02b542f"]
512026-03-24T18:10:14.135ZINFOcrucible-agent (dropshot): request completed latency_us = 19055 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43632 req_id = 2bdee1d2-5f63-4379-ac9a-28412885284e response_code = 200 uri = /crucible/0/regions/3f796b11-b103-4585-aee9-a1878b120734/snapshots/daf70307-7db2-49dc-ada4-0c7ea02b542f/run