Max level shown:
12026-08-05T05:18:33.773ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::b]:56454
22026-08-05T05:18:33.773ZINFOcrucible-agent (datafile): region 18ce9039-0d39-442c-bdf4-e60598a53717 state: Requested
32026-08-05T05:18:33.774ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-08-05T05:18:33.774ZINFOcrucible-agent (dropshot): request completed latency_us = 723 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:56454 req_id = 75fed2a4-750b-4ab3-8505-eed75e9d23fd response_code = 200 uri = /crucible/0/regions
52026-08-05T05:18:33.783ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/18ce9039-0d39-442c-bdf4-e60598a53717
62026-08-05T05:18:33.783ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/18ce9039-0d39-442c-bdf4-e60598a53717
72026-08-05T05:18:33.824ZINFOcrucible-agent (worker): creating region Region { id: RegionId("18ce9039-0d39-442c-bdf4-e60598a53717"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/18ce9039-0d39-442c-bdf4-e60598a53717" region = 18ce9039-0d39-442c-bdf4-e60598a53717
82026-08-05T05:18:33.824ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "18ce9039-0d39-442c-bdf4-e60598a53717" "--data" "/data/regions/18ce9039-0d39-442c-bdf4-e60598a53717" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 18ce9039-0d39-442c-bdf4-e60598a53717
92026-08-05T05:18:33.878ZINFOcrucible-agent (worker): region files created ok region = 18ce9039-0d39-442c-bdf4-e60598a53717
102026-08-05T05:18:33.878ZINFOcrucible-agent (datafile): region 18ce9039-0d39-442c-bdf4-e60598a53717 state: Requested -> Created
112026-08-05T05:18:33.878ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-08-05T05:18:33.879ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-18ce9039-0d39-442c-bdf4-e60598a53717
132026-08-05T05:18:33.895ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-18ce9039-0d39-442c-bdf4-e60598a53717
142026-08-05T05:18:33.903ZINFOcrucible-agent (worker): creating config property group
152026-08-05T05:18:33.904ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-18ce9039-0d39-442c-bdf4-e60598a53717
162026-08-05T05:18:33.904ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/18ce9039-0d39-442c-bdf4-e60598a53717
172026-08-05T05:18:33.904ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-08-05T05:18:33.904ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
192026-08-05T05:18:33.905ZINFOcrucible-agent (worker): commit
202026-08-05T05:18:33.906ZINFOcrucible-agent (worker): ok!
212026-08-05T05:18:33.910ZINFOcrucible-agent (worker): SMF ok!
222026-08-05T05:18:33.952ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:56454 req_id = 6a3d12e9-0f62-4ee0-8f96-6e6c81661be9 response_code = 200 uri = /crucible/0/regions
232026-08-05T05:18:35.706ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:56454 req_id = ed4152eb-0b79-455f-acb8-d25d51bc04bc response_code = 200 uri = /crucible/0/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9
242026-08-05T05:18:35.709ZINFOcrucible-agent (datafile): path is "/data/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9"
252026-08-05T05:18:35.715ZINFOcrucible-agent (datafile): dataset is oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9
262026-08-05T05:18:35.721ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9@b62b5337-9c23-497d-99e7-6e8a97e3df3f
272026-08-05T05:18:35.721ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9", "b62b5337-9c23-497d-99e7-6e8a97e3df3f"]
282026-08-05T05:18:35.733ZINFOcrucible-agent (dropshot): request completed latency_us = 23912 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:56454 req_id = 0c5c4ce7-5ef4-4c85-ab96-8d742819a31a response_code = 200 uri = /crucible/0/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9/snapshots/b62b5337-9c23-497d-99e7-6e8a97e3df3f
292026-08-05T05:18:35.737ZINFOcrucible-agent (datafile): path is "/data/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9"
302026-08-05T05:18:35.743ZINFOcrucible-agent (datafile): dataset is oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9
312026-08-05T05:18:35.749ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9@b62b5337-9c23-497d-99e7-6e8a97e3df3f
322026-08-05T05:18:35.749ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9", "b62b5337-9c23-497d-99e7-6e8a97e3df3f"]
332026-08-05T05:18:35.755ZINFOcrucible-agent (datafile): requesting running snapshot 8a25dae9-8515-4941-a963-a3b7ad78f6b9-b62b5337-9c23-497d-99e7-6e8a97e3df3f state: Requested
342026-08-05T05:18:35.760ZINFOcrucible-agent (worker): applying SMF actions for region 8a25dae9-8515-4941-a963-a3b7ad78f6b9 running snapshot b62b5337-9c23-497d-99e7-6e8a97e3df3f (state Requested)...
352026-08-05T05:18:35.760ZINFOcrucible-agent (dropshot): request completed latency_us = 23457 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:56454 req_id = 81e2a60c-6660-44c0-846b-b74996545d41 response_code = 200 uri = /crucible/0/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9/snapshots/b62b5337-9c23-497d-99e7-6e8a97e3df3f/run
362026-08-05T05:18:35.763ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-8a25dae9-8515-4941-a963-a3b7ad78f6b9-b62b5337-9c23-497d-99e7-6e8a97e3df3f
372026-08-05T05:18:35.764ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-8a25dae9-8515-4941-a963-a3b7ad78f6b9-b62b5337-9c23-497d-99e7-6e8a97e3df3f
382026-08-05T05:18:35.770ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-8a25dae9-8515-4941-a963-a3b7ad78f6b9-b62b5337-9c23-497d-99e7-6e8a97e3df3f
392026-08-05T05:18:35.770ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9/.zfs/snapshot/b62b5337-9c23-497d-99e7-6e8a97e3df3f
402026-08-05T05:18:35.770ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-08-05T05:18:35.770ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-08-05T05:18:35.770ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
432026-08-05T05:18:35.770ZINFOcrucible-agent (worker): commit
442026-08-05T05:18:35.772ZINFOcrucible-agent (worker): ok!
452026-08-05T05:18:35.775ZINFOcrucible-agent (worker): SMF ok!
462026-08-05T05:18:35.775ZINFOcrucible-agent (datafile): region 8a25dae9-8515-4941-a963-a3b7ad78f6b9 running snapshot b62b5337-9c23-497d-99e7-6e8a97e3df3f state: Requested -> Created
472026-08-05T05:18:35.994ZINFOcrucible-agent (datafile): path is "/data/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9"
482026-08-05T05:18:36.001ZINFOcrucible-agent (datafile): dataset is oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9
492026-08-05T05:18:36.007ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9@b62b5337-9c23-497d-99e7-6e8a97e3df3f
502026-08-05T05:18:36.007ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b8c14b16-d5cb-40d0-a960-15cb937c3800/crucible/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9", "b62b5337-9c23-497d-99e7-6e8a97e3df3f"]
512026-08-05T05:18:36.012ZINFOcrucible-agent (dropshot): request completed latency_us = 17460 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:56454 req_id = 0681b5fc-e6fc-43e3-b75f-a0d95f8c1329 response_code = 200 uri = /crucible/0/regions/8a25dae9-8515-4941-a963-a3b7ad78f6b9/snapshots/b62b5337-9c23-497d-99e7-6e8a97e3df3f/run