Max level shown:
12026-08-11T21:55:03.729ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::c]:39467
22026-08-11T21:55:03.729ZINFOcrucible-agent (datafile): region e4422d6c-352f-4641-bed8-5bcc258458a9 state: Requested
32026-08-11T21:55:03.730ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-08-11T21:55:03.730ZINFOcrucible-agent (dropshot): request completed latency_us = 720 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39467 req_id = 9fe77fc7-56ba-4325-93c7-7fa728c9c5f1 response_code = 200 uri = /crucible/0/regions
52026-08-11T21:55:03.741ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/e4422d6c-352f-4641-bed8-5bcc258458a9
62026-08-11T21:55:03.741ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/e4422d6c-352f-4641-bed8-5bcc258458a9
72026-08-11T21:55:03.799ZINFOcrucible-agent (worker): creating region Region { id: RegionId("e4422d6c-352f-4641-bed8-5bcc258458a9"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/e4422d6c-352f-4641-bed8-5bcc258458a9" region = e4422d6c-352f-4641-bed8-5bcc258458a9
82026-08-11T21:55:03.799ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "e4422d6c-352f-4641-bed8-5bcc258458a9" "--data" "/data/regions/e4422d6c-352f-4641-bed8-5bcc258458a9" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = e4422d6c-352f-4641-bed8-5bcc258458a9
92026-08-11T21:55:03.875ZINFOcrucible-agent (worker): region files created ok region = e4422d6c-352f-4641-bed8-5bcc258458a9
102026-08-11T21:55:03.875ZINFOcrucible-agent (datafile): region e4422d6c-352f-4641-bed8-5bcc258458a9 state: Requested -> Created
112026-08-11T21:55:03.875ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-08-11T21:55:03.882ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-e4422d6c-352f-4641-bed8-5bcc258458a9
132026-08-11T21:55:03.890ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-e4422d6c-352f-4641-bed8-5bcc258458a9
142026-08-11T21:55:03.900ZINFOcrucible-agent (worker): creating config property group
152026-08-11T21:55:03.903ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-e4422d6c-352f-4641-bed8-5bcc258458a9
162026-08-11T21:55:03.904ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e4422d6c-352f-4641-bed8-5bcc258458a9
172026-08-11T21:55:03.904ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-08-11T21:55:03.904ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
192026-08-11T21:55:03.904ZINFOcrucible-agent (worker): commit
202026-08-11T21:55:03.907ZINFOcrucible-agent (worker): ok!
212026-08-11T21:55:03.912ZINFOcrucible-agent (worker): SMF ok!
222026-08-11T21:55:03.935ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39467 req_id = 2e8914c3-35a4-4ab4-884c-ebd9f3f60b7e response_code = 200 uri = /crucible/0/regions
232026-08-11T21:55:05.224ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:39467 req_id = 6ad69cd6-32c9-440a-a9d0-9cdfd450781b response_code = 200 uri = /crucible/0/regions/16014196-d9b2-422a-a0be-91bb25d6cab6
242026-08-11T21:55:05.228ZINFOcrucible-agent (datafile): path is "/data/regions/16014196-d9b2-422a-a0be-91bb25d6cab6"
252026-08-11T21:55:05.236ZINFOcrucible-agent (datafile): dataset is oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/16014196-d9b2-422a-a0be-91bb25d6cab6
262026-08-11T21:55:05.242ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/16014196-d9b2-422a-a0be-91bb25d6cab6@8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e
272026-08-11T21:55:05.242ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/16014196-d9b2-422a-a0be-91bb25d6cab6", "8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e"]
282026-08-11T21:55:05.253ZINFOcrucible-agent (dropshot): request completed latency_us = 25081 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:39467 req_id = 0ddfb079-4c1e-4bf7-8d07-d4916e8af49c response_code = 200 uri = /crucible/0/regions/16014196-d9b2-422a-a0be-91bb25d6cab6/snapshots/8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e
292026-08-11T21:55:05.256ZINFOcrucible-agent (datafile): path is "/data/regions/16014196-d9b2-422a-a0be-91bb25d6cab6"
302026-08-11T21:55:05.262ZINFOcrucible-agent (datafile): dataset is oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/16014196-d9b2-422a-a0be-91bb25d6cab6
312026-08-11T21:55:05.267ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/16014196-d9b2-422a-a0be-91bb25d6cab6@8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e
322026-08-11T21:55:05.267ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/16014196-d9b2-422a-a0be-91bb25d6cab6", "8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e"]
332026-08-11T21:55:05.273ZINFOcrucible-agent (datafile): requesting running snapshot 16014196-d9b2-422a-a0be-91bb25d6cab6-8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e state: Requested
342026-08-11T21:55:05.277ZINFOcrucible-agent (worker): applying SMF actions for region 16014196-d9b2-422a-a0be-91bb25d6cab6 running snapshot 8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e (state Requested)...
352026-08-11T21:55:05.277ZINFOcrucible-agent (dropshot): request completed latency_us = 20820 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39467 req_id = 6d3163de-6061-4fa4-b811-5fb2fee9863e response_code = 200 uri = /crucible/0/regions/16014196-d9b2-422a-a0be-91bb25d6cab6/snapshots/8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e/run
362026-08-11T21:55:05.280ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-16014196-d9b2-422a-a0be-91bb25d6cab6-8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e
372026-08-11T21:55:05.282ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-16014196-d9b2-422a-a0be-91bb25d6cab6-8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e
382026-08-11T21:55:05.292ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-16014196-d9b2-422a-a0be-91bb25d6cab6-8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e
392026-08-11T21:55:05.293ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/16014196-d9b2-422a-a0be-91bb25d6cab6/.zfs/snapshot/8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e
402026-08-11T21:55:05.293ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-08-11T21:55:05.293ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-08-11T21:55:05.293ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432026-08-11T21:55:05.293ZINFOcrucible-agent (worker): commit
442026-08-11T21:55:05.296ZINFOcrucible-agent (worker): ok!
452026-08-11T21:55:05.301ZINFOcrucible-agent (worker): SMF ok!
462026-08-11T21:55:05.301ZINFOcrucible-agent (datafile): region 16014196-d9b2-422a-a0be-91bb25d6cab6 running snapshot 8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e state: Requested -> Created
472026-08-11T21:55:05.484ZINFOcrucible-agent (datafile): path is "/data/regions/16014196-d9b2-422a-a0be-91bb25d6cab6"
482026-08-11T21:55:05.490ZINFOcrucible-agent (datafile): dataset is oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/16014196-d9b2-422a-a0be-91bb25d6cab6
492026-08-11T21:55:05.495ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/16014196-d9b2-422a-a0be-91bb25d6cab6@8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e
502026-08-11T21:55:05.495ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_69eab91f-f314-44ce-9730-bf99577e5a48/crucible/regions/16014196-d9b2-422a-a0be-91bb25d6cab6", "8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e"]
512026-08-11T21:55:05.500ZINFOcrucible-agent (dropshot): request completed latency_us = 16128 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39467 req_id = aebd89dc-3e69-40e4-aa3b-948cb80ac255 response_code = 200 uri = /crucible/0/regions/16014196-d9b2-422a-a0be-91bb25d6cab6/snapshots/8ddd1e2f-85f5-49ee-ad5c-a1a838e6990e/run