Max level shown:
12026-01-24T01:05:08.237ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::c]:44060
22026-01-24T01:05:08.237ZINFOcrucible-agent (datafile): region 830c52d7-2292-426e-824a-84b9afc49407 state: Requested
32026-01-24T01:05:08.242ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-01-24T01:05:08.242ZINFOcrucible-agent (dropshot): request completed latency_us = 4646 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44060 req_id = da6930b5-cce3-4508-9f47-0681ffa90394 response_code = 200 uri = /crucible/0/regions
52026-01-24T01:05:08.253ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/830c52d7-2292-426e-824a-84b9afc49407
62026-01-24T01:05:08.253ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/830c52d7-2292-426e-824a-84b9afc49407
72026-01-24T01:05:08.421ZINFOcrucible-agent (worker): creating region Region { id: RegionId("830c52d7-2292-426e-824a-84b9afc49407"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/830c52d7-2292-426e-824a-84b9afc49407" region = 830c52d7-2292-426e-824a-84b9afc49407
82026-01-24T01:05:08.421ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "830c52d7-2292-426e-824a-84b9afc49407" "--data" "/data/regions/830c52d7-2292-426e-824a-84b9afc49407" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 830c52d7-2292-426e-824a-84b9afc49407
92026-01-24T01:05:08.467ZINFOcrucible-agent (dropshot): request completed latency_us = 50 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44060 req_id = f743384c-13c2-4602-9ca9-13fceab826f1 response_code = 200 uri = /crucible/0/regions
102026-01-24T01:05:08.744ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44060 req_id = b16e3700-478d-4e79-b80e-0783bf809c7a response_code = 200 uri = /crucible/0/regions
112026-01-24T01:05:08.794ZINFOcrucible-agent (worker): region files created ok region = 830c52d7-2292-426e-824a-84b9afc49407
122026-01-24T01:05:08.794ZINFOcrucible-agent (datafile): region 830c52d7-2292-426e-824a-84b9afc49407 state: Requested -> Created
132026-01-24T01:05:08.795ZINFOcrucible-agent (worker): applying SMF actions post create...
142026-01-24T01:05:08.829ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-830c52d7-2292-426e-824a-84b9afc49407
152026-01-24T01:05:08.888ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-830c52d7-2292-426e-824a-84b9afc49407
162026-01-24T01:05:08.924ZINFOcrucible-agent (worker): creating config property group
172026-01-24T01:05:08.946ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-830c52d7-2292-426e-824a-84b9afc49407
182026-01-24T01:05:08.947ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/830c52d7-2292-426e-824a-84b9afc49407
192026-01-24T01:05:08.947ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
202026-01-24T01:05:08.947ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
212026-01-24T01:05:08.947ZINFOcrucible-agent (worker): commit
222026-01-24T01:05:08.970ZINFOcrucible-agent (worker): ok!
232026-01-24T01:05:08.982ZINFOcrucible-agent (worker): SMF ok!
242026-01-24T01:05:10.071ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44060 req_id = f5c5b842-e3d3-4aa0-a6c3-01455486bc92 response_code = 200 uri = /crucible/0/regions
252026-01-24T01:05:11.192ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:44060 req_id = 3e0cba84-f97f-4429-ab37-eddd33b89159 response_code = 200 uri = /crucible/0/regions/20290dc6-e974-4db2-a1ec-644133e1c88a
262026-01-24T01:05:11.203ZINFOcrucible-agent (datafile): path is "/data/regions/20290dc6-e974-4db2-a1ec-644133e1c88a"
272026-01-24T01:05:11.209ZINFOcrucible-agent (datafile): dataset is oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/20290dc6-e974-4db2-a1ec-644133e1c88a
282026-01-24T01:05:11.217ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/20290dc6-e974-4db2-a1ec-644133e1c88a@fa3dff89-d21d-4076-a16c-92032f16992c
292026-01-24T01:05:11.217ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/20290dc6-e974-4db2-a1ec-644133e1c88a", "fa3dff89-d21d-4076-a16c-92032f16992c"]
302026-01-24T01:05:11.226ZINFOcrucible-agent (dropshot): request completed latency_us = 22997 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:44060 req_id = c6b0da56-b462-41d1-8ced-aa5642fb5d0b response_code = 200 uri = /crucible/0/regions/20290dc6-e974-4db2-a1ec-644133e1c88a/snapshots/fa3dff89-d21d-4076-a16c-92032f16992c
312026-01-24T01:05:11.233ZINFOcrucible-agent (datafile): path is "/data/regions/20290dc6-e974-4db2-a1ec-644133e1c88a"
322026-01-24T01:05:11.240ZINFOcrucible-agent (datafile): dataset is oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/20290dc6-e974-4db2-a1ec-644133e1c88a
332026-01-24T01:05:11.246ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/20290dc6-e974-4db2-a1ec-644133e1c88a@fa3dff89-d21d-4076-a16c-92032f16992c
342026-01-24T01:05:11.246ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/20290dc6-e974-4db2-a1ec-644133e1c88a", "fa3dff89-d21d-4076-a16c-92032f16992c"]
352026-01-24T01:05:11.259ZINFOcrucible-agent (datafile): requesting running snapshot 20290dc6-e974-4db2-a1ec-644133e1c88a-fa3dff89-d21d-4076-a16c-92032f16992c state: Requested
362026-01-24T01:05:11.259ZINFOcrucible-agent (dropshot): request completed latency_us = 26025 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44060 req_id = 8f43eb5c-5bfe-4389-b77c-6bcd728f41a3 response_code = 200 uri = /crucible/0/regions/20290dc6-e974-4db2-a1ec-644133e1c88a/snapshots/fa3dff89-d21d-4076-a16c-92032f16992c/run
372026-01-24T01:05:11.259ZINFOcrucible-agent (worker): applying SMF actions for region 20290dc6-e974-4db2-a1ec-644133e1c88a running snapshot fa3dff89-d21d-4076-a16c-92032f16992c (state Requested)...
382026-01-24T01:05:11.264ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-20290dc6-e974-4db2-a1ec-644133e1c88a-fa3dff89-d21d-4076-a16c-92032f16992c
392026-01-24T01:05:11.268ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-20290dc6-e974-4db2-a1ec-644133e1c88a-fa3dff89-d21d-4076-a16c-92032f16992c
402026-01-24T01:05:11.298ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-20290dc6-e974-4db2-a1ec-644133e1c88a-fa3dff89-d21d-4076-a16c-92032f16992c
412026-01-24T01:05:11.298ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/20290dc6-e974-4db2-a1ec-644133e1c88a/.zfs/snapshot/fa3dff89-d21d-4076-a16c-92032f16992c
422026-01-24T01:05:11.299ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
432026-01-24T01:05:11.299ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
442026-01-24T01:05:11.299ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
452026-01-24T01:05:11.299ZINFOcrucible-agent (worker): commit
462026-01-24T01:05:11.303ZINFOcrucible-agent (worker): ok!
472026-01-24T01:05:11.317ZINFOcrucible-agent (worker): SMF ok!
482026-01-24T01:05:11.317ZINFOcrucible-agent (datafile): region 20290dc6-e974-4db2-a1ec-644133e1c88a running snapshot fa3dff89-d21d-4076-a16c-92032f16992c state: Requested -> Created
492026-01-24T01:05:11.446ZINFOcrucible-agent (datafile): path is "/data/regions/20290dc6-e974-4db2-a1ec-644133e1c88a"
502026-01-24T01:05:11.453ZINFOcrucible-agent (datafile): dataset is oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/20290dc6-e974-4db2-a1ec-644133e1c88a
512026-01-24T01:05:11.460ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/20290dc6-e974-4db2-a1ec-644133e1c88a@fa3dff89-d21d-4076-a16c-92032f16992c
522026-01-24T01:05:11.460ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ad7087ea-1bc0-4cb4-bc9f-553015ef0f6c/crucible/regions/20290dc6-e974-4db2-a1ec-644133e1c88a", "fa3dff89-d21d-4076-a16c-92032f16992c"]
532026-01-24T01:05:11.466ZINFOcrucible-agent (dropshot): request completed latency_us = 19523 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:44060 req_id = 59f57d87-f69c-4a33-b151-451307f83216 response_code = 200 uri = /crucible/0/regions/20290dc6-e974-4db2-a1ec-644133e1c88a/snapshots/fa3dff89-d21d-4076-a16c-92032f16992c/run