Max level shown:
12026-09-04T16:05:34.516ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:33606
22026-09-04T16:05:34.516ZINFOcrucible-agent (datafile): region 20778174-e35e-4de2-aff7-e9c54141f3f9 state: Requested
32026-09-04T16:05:34.517ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-09-04T16:05:34.517ZINFOcrucible-agent (dropshot): request completed latency_us = 777 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33606 req_id = 76e4a04c-cc22-48e4-b994-072e70721314 response_code = 200 uri = /crucible/0/regions
52026-09-04T16:05:34.526ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/20778174-e35e-4de2-aff7-e9c54141f3f9
62026-09-04T16:05:34.526ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/20778174-e35e-4de2-aff7-e9c54141f3f9
72026-09-04T16:05:34.581ZINFOcrucible-agent (worker): creating region Region { id: RegionId("20778174-e35e-4de2-aff7-e9c54141f3f9"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/20778174-e35e-4de2-aff7-e9c54141f3f9" region = 20778174-e35e-4de2-aff7-e9c54141f3f9
82026-09-04T16:05:34.582ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "20778174-e35e-4de2-aff7-e9c54141f3f9" "--data" "/data/regions/20778174-e35e-4de2-aff7-e9c54141f3f9" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 20778174-e35e-4de2-aff7-e9c54141f3f9
92026-09-04T16:05:34.698ZINFOcrucible-agent (worker): region files created ok region = 20778174-e35e-4de2-aff7-e9c54141f3f9
102026-09-04T16:05:34.698ZINFOcrucible-agent (datafile): region 20778174-e35e-4de2-aff7-e9c54141f3f9 state: Requested -> Created
112026-09-04T16:05:34.698ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-09-04T16:05:34.699ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-20778174-e35e-4de2-aff7-e9c54141f3f9
132026-09-04T16:05:34.708ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-20778174-e35e-4de2-aff7-e9c54141f3f9
142026-09-04T16:05:34.731ZINFOcrucible-agent (worker): creating config property group
152026-09-04T16:05:34.736ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-20778174-e35e-4de2-aff7-e9c54141f3f9
162026-09-04T16:05:34.737ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/20778174-e35e-4de2-aff7-e9c54141f3f9
172026-09-04T16:05:34.737ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-09-04T16:05:34.737ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
192026-09-04T16:05:34.737ZINFOcrucible-agent (worker): commit
202026-09-04T16:05:34.745ZINFOcrucible-agent (worker): ok!
212026-09-04T16:05:34.756ZINFOcrucible-agent (worker): SMF ok!
222026-09-04T16:05:34.891ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33606 req_id = c92c6c30-a900-4272-b1ea-263268973368 response_code = 200 uri = /crucible/0/regions
232026-09-04T16:05:35.906ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33606 req_id = e1e1b80a-153e-4a8a-8d15-807ea291279c response_code = 200 uri = /crucible/0/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0
242026-09-04T16:05:35.910ZINFOcrucible-agent (datafile): path is "/data/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0"
252026-09-04T16:05:35.916ZINFOcrucible-agent (datafile): dataset is oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0
262026-09-04T16:05:35.923ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0@1d550e20-d99d-4434-bc59-da3caaad2b0c
272026-09-04T16:05:35.923ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0", "1d550e20-d99d-4434-bc59-da3caaad2b0c"]
282026-09-04T16:05:35.935ZINFOcrucible-agent (dropshot): request completed latency_us = 25468 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33606 req_id = 42f32ccc-53f7-4ffe-b348-6c10fe12e979 response_code = 200 uri = /crucible/0/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0/snapshots/1d550e20-d99d-4434-bc59-da3caaad2b0c
292026-09-04T16:05:35.944ZINFOcrucible-agent (datafile): path is "/data/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0"
302026-09-04T16:05:35.952ZINFOcrucible-agent (datafile): dataset is oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0
312026-09-04T16:05:35.958ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0@1d550e20-d99d-4434-bc59-da3caaad2b0c
322026-09-04T16:05:35.958ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0", "1d550e20-d99d-4434-bc59-da3caaad2b0c"]
332026-09-04T16:05:35.965ZINFOcrucible-agent (datafile): requesting running snapshot 704fde31-beab-4ec4-b4d8-b3cfbdad29d0-1d550e20-d99d-4434-bc59-da3caaad2b0c state: Requested
342026-09-04T16:05:35.970ZINFOcrucible-agent (worker): applying SMF actions for region 704fde31-beab-4ec4-b4d8-b3cfbdad29d0 running snapshot 1d550e20-d99d-4434-bc59-da3caaad2b0c (state Requested)...
352026-09-04T16:05:35.970ZINFOcrucible-agent (dropshot): request completed latency_us = 25987 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33606 req_id = 7b4d9a49-c7c2-4818-8bdc-4e55fb05be80 response_code = 200 uri = /crucible/0/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0/snapshots/1d550e20-d99d-4434-bc59-da3caaad2b0c/run
362026-09-04T16:05:35.972ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-704fde31-beab-4ec4-b4d8-b3cfbdad29d0-1d550e20-d99d-4434-bc59-da3caaad2b0c
372026-09-04T16:05:35.983ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-704fde31-beab-4ec4-b4d8-b3cfbdad29d0-1d550e20-d99d-4434-bc59-da3caaad2b0c
382026-09-04T16:05:35.993ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-704fde31-beab-4ec4-b4d8-b3cfbdad29d0-1d550e20-d99d-4434-bc59-da3caaad2b0c
392026-09-04T16:05:35.993ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0/.zfs/snapshot/1d550e20-d99d-4434-bc59-da3caaad2b0c
402026-09-04T16:05:35.993ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-09-04T16:05:35.993ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-09-04T16:05:35.993ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
432026-09-04T16:05:35.993ZINFOcrucible-agent (worker): commit
442026-09-04T16:05:35.997ZINFOcrucible-agent (worker): ok!
452026-09-04T16:05:36.001ZINFOcrucible-agent (worker): SMF ok!
462026-09-04T16:05:36.001ZINFOcrucible-agent (datafile): region 704fde31-beab-4ec4-b4d8-b3cfbdad29d0 running snapshot 1d550e20-d99d-4434-bc59-da3caaad2b0c state: Requested -> Created
472026-09-04T16:05:36.229ZINFOcrucible-agent (datafile): path is "/data/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0"
482026-09-04T16:05:36.237ZINFOcrucible-agent (datafile): dataset is oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0
492026-09-04T16:05:36.242ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0@1d550e20-d99d-4434-bc59-da3caaad2b0c
502026-09-04T16:05:36.242ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e5ba7ab2-71af-45f7-a6f5-0771792005f4/crucible/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0", "1d550e20-d99d-4434-bc59-da3caaad2b0c"]
512026-09-04T16:05:36.249ZINFOcrucible-agent (dropshot): request completed latency_us = 20213 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:33606 req_id = f418b871-4379-4898-8d5f-f52e452c3ccc response_code = 200 uri = /crucible/0/regions/704fde31-beab-4ec4-b4d8-b3cfbdad29d0/snapshots/1d550e20-d99d-4434-bc59-da3caaad2b0c/run