Max level shown:
12026-01-17T04:46:16.346ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:43455
22026-01-17T04:46:16.346ZINFOcrucible-agent (datafile): region 968469c4-0aaf-4822-a24c-3e3df1c17f57 state: Requested
32026-01-17T04:46:16.347ZINFOcrucible-agent (dropshot): request completed latency_us = 644 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43455 req_id = 27222db8-1fad-457e-a27b-9df1e6b998f4 response_code = 200 uri = /crucible/0/regions
42026-01-17T04:46:16.347ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-01-17T04:46:16.356ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/968469c4-0aaf-4822-a24c-3e3df1c17f57
62026-01-17T04:46:16.356ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/968469c4-0aaf-4822-a24c-3e3df1c17f57
72026-01-17T04:46:16.413ZINFOcrucible-agent (worker): creating region Region { id: RegionId("968469c4-0aaf-4822-a24c-3e3df1c17f57"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/968469c4-0aaf-4822-a24c-3e3df1c17f57" region = 968469c4-0aaf-4822-a24c-3e3df1c17f57
82026-01-17T04:46:16.413ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "968469c4-0aaf-4822-a24c-3e3df1c17f57" "--data" "/data/regions/968469c4-0aaf-4822-a24c-3e3df1c17f57" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 968469c4-0aaf-4822-a24c-3e3df1c17f57
92026-01-17T04:46:16.512ZINFOcrucible-agent (worker): region files created ok region = 968469c4-0aaf-4822-a24c-3e3df1c17f57
102026-01-17T04:46:16.512ZINFOcrucible-agent (datafile): region 968469c4-0aaf-4822-a24c-3e3df1c17f57 state: Requested -> Created
112026-01-17T04:46:16.512ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-01-17T04:46:16.513ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-968469c4-0aaf-4822-a24c-3e3df1c17f57
132026-01-17T04:46:16.524ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-968469c4-0aaf-4822-a24c-3e3df1c17f57
142026-01-17T04:46:16.546ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43455 req_id = 0401d375-6ce5-470e-be3f-b40d978c828c response_code = 200 uri = /crucible/0/regions
152026-01-17T04:46:16.547ZINFOcrucible-agent (worker): creating config property group
162026-01-17T04:46:16.550ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-968469c4-0aaf-4822-a24c-3e3df1c17f57
172026-01-17T04:46:16.551ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/968469c4-0aaf-4822-a24c-3e3df1c17f57
182026-01-17T04:46:16.551ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192026-01-17T04:46:16.551ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
202026-01-17T04:46:16.551ZINFOcrucible-agent (worker): commit
212026-01-17T04:46:16.555ZINFOcrucible-agent (worker): ok!
222026-01-17T04:46:16.604ZINFOcrucible-agent (worker): SMF ok!
232026-01-17T04:46:17.783ZINFOcrucible-agent (dropshot): request completed latency_us = 89 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43455 req_id = 4ab5f589-35ae-496c-993d-24bb1d40af0c response_code = 200 uri = /crucible/0/regions/96989055-eac0-4ce7-b850-c377650d5ed7
242026-01-17T04:46:17.791ZINFOcrucible-agent (datafile): path is "/data/regions/96989055-eac0-4ce7-b850-c377650d5ed7"
252026-01-17T04:46:17.797ZINFOcrucible-agent (datafile): dataset is oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/96989055-eac0-4ce7-b850-c377650d5ed7
262026-01-17T04:46:17.804ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/96989055-eac0-4ce7-b850-c377650d5ed7@2488f8d4-0e4a-4432-a0ed-646d1e3213ae
272026-01-17T04:46:17.804ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/96989055-eac0-4ce7-b850-c377650d5ed7", "2488f8d4-0e4a-4432-a0ed-646d1e3213ae"]
282026-01-17T04:46:17.812ZINFOcrucible-agent (dropshot): request completed latency_us = 21776 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43455 req_id = a413d738-3586-4427-a601-81771086a9b8 response_code = 200 uri = /crucible/0/regions/96989055-eac0-4ce7-b850-c377650d5ed7/snapshots/2488f8d4-0e4a-4432-a0ed-646d1e3213ae
292026-01-17T04:46:17.818ZINFOcrucible-agent (datafile): path is "/data/regions/96989055-eac0-4ce7-b850-c377650d5ed7"
302026-01-17T04:46:17.824ZINFOcrucible-agent (datafile): dataset is oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/96989055-eac0-4ce7-b850-c377650d5ed7
312026-01-17T04:46:17.830ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/96989055-eac0-4ce7-b850-c377650d5ed7@2488f8d4-0e4a-4432-a0ed-646d1e3213ae
322026-01-17T04:46:17.830ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/96989055-eac0-4ce7-b850-c377650d5ed7", "2488f8d4-0e4a-4432-a0ed-646d1e3213ae"]
332026-01-17T04:46:17.839ZINFOcrucible-agent (datafile): requesting running snapshot 96989055-eac0-4ce7-b850-c377650d5ed7-2488f8d4-0e4a-4432-a0ed-646d1e3213ae state: Requested
342026-01-17T04:46:17.839ZINFOcrucible-agent (dropshot): request completed latency_us = 21352 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43455 req_id = 742f12a3-25d1-482c-8822-1da2ef3ed5f1 response_code = 200 uri = /crucible/0/regions/96989055-eac0-4ce7-b850-c377650d5ed7/snapshots/2488f8d4-0e4a-4432-a0ed-646d1e3213ae/run
352026-01-17T04:46:17.839ZINFOcrucible-agent (worker): applying SMF actions for region 96989055-eac0-4ce7-b850-c377650d5ed7 running snapshot 2488f8d4-0e4a-4432-a0ed-646d1e3213ae (state Requested)...
362026-01-17T04:46:17.844ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-96989055-eac0-4ce7-b850-c377650d5ed7-2488f8d4-0e4a-4432-a0ed-646d1e3213ae
372026-01-17T04:46:17.852ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-96989055-eac0-4ce7-b850-c377650d5ed7-2488f8d4-0e4a-4432-a0ed-646d1e3213ae
382026-01-17T04:46:17.862ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-96989055-eac0-4ce7-b850-c377650d5ed7-2488f8d4-0e4a-4432-a0ed-646d1e3213ae
392026-01-17T04:46:17.862ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/96989055-eac0-4ce7-b850-c377650d5ed7/.zfs/snapshot/2488f8d4-0e4a-4432-a0ed-646d1e3213ae
402026-01-17T04:46:17.862ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-01-17T04:46:17.862ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-01-17T04:46:17.863ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
432026-01-17T04:46:17.863ZINFOcrucible-agent (worker): commit
442026-01-17T04:46:17.867ZINFOcrucible-agent (worker): ok!
452026-01-17T04:46:17.872ZINFOcrucible-agent (worker): SMF ok!
462026-01-17T04:46:17.872ZINFOcrucible-agent (datafile): region 96989055-eac0-4ce7-b850-c377650d5ed7 running snapshot 2488f8d4-0e4a-4432-a0ed-646d1e3213ae state: Requested -> Created
472026-01-17T04:46:18.087ZINFOcrucible-agent (datafile): path is "/data/regions/96989055-eac0-4ce7-b850-c377650d5ed7"
482026-01-17T04:46:18.093ZINFOcrucible-agent (datafile): dataset is oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/96989055-eac0-4ce7-b850-c377650d5ed7
492026-01-17T04:46:18.100ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/96989055-eac0-4ce7-b850-c377650d5ed7@2488f8d4-0e4a-4432-a0ed-646d1e3213ae
502026-01-17T04:46:18.100ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_27f9684b-1fd9-4fb8-a8ae-a7f86a18024a/crucible/regions/96989055-eac0-4ce7-b850-c377650d5ed7", "2488f8d4-0e4a-4432-a0ed-646d1e3213ae"]
512026-01-17T04:46:18.107ZINFOcrucible-agent (dropshot): request completed latency_us = 20409 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43455 req_id = 57d0c131-df66-44a6-8717-00ae47537fec response_code = 200 uri = /crucible/0/regions/96989055-eac0-4ce7-b850-c377650d5ed7/snapshots/2488f8d4-0e4a-4432-a0ed-646d1e3213ae/run