Max level shown:
12025-12-18T04:05:09.378ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::c]:50069
22025-12-18T04:05:09.378ZINFOcrucible-agent (datafile): region 974af7fa-bc0a-4b19-88c4-502b8d9a8079 state: Requested
32025-12-18T04:05:09.378ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-12-18T04:05:09.378ZINFOcrucible-agent (dropshot): request completed latency_us = 662 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50069 req_id = bd7c8c7a-a630-47b3-b680-b77d359b193b response_code = 200 uri = /crucible/0/regions
52025-12-18T04:05:09.388ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/974af7fa-bc0a-4b19-88c4-502b8d9a8079
62025-12-18T04:05:09.388ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/974af7fa-bc0a-4b19-88c4-502b8d9a8079
72025-12-18T04:05:09.447ZINFOcrucible-agent (worker): creating region Region { id: RegionId("974af7fa-bc0a-4b19-88c4-502b8d9a8079"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/974af7fa-bc0a-4b19-88c4-502b8d9a8079" region = 974af7fa-bc0a-4b19-88c4-502b8d9a8079
82025-12-18T04:05:09.447ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "974af7fa-bc0a-4b19-88c4-502b8d9a8079" "--data" "/data/regions/974af7fa-bc0a-4b19-88c4-502b8d9a8079" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 974af7fa-bc0a-4b19-88c4-502b8d9a8079
92025-12-18T04:05:09.523ZINFOcrucible-agent (worker): region files created ok region = 974af7fa-bc0a-4b19-88c4-502b8d9a8079
102025-12-18T04:05:09.523ZINFOcrucible-agent (datafile): region 974af7fa-bc0a-4b19-88c4-502b8d9a8079 state: Requested -> Created
112025-12-18T04:05:09.523ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-12-18T04:05:09.523ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-974af7fa-bc0a-4b19-88c4-502b8d9a8079
132025-12-18T04:05:09.533ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-974af7fa-bc0a-4b19-88c4-502b8d9a8079
142025-12-18T04:05:09.554ZINFOcrucible-agent (worker): creating config property group
152025-12-18T04:05:09.557ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-974af7fa-bc0a-4b19-88c4-502b8d9a8079
162025-12-18T04:05:09.557ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/974af7fa-bc0a-4b19-88c4-502b8d9a8079
172025-12-18T04:05:09.557ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-12-18T04:05:09.557ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
192025-12-18T04:05:09.557ZINFOcrucible-agent (worker): commit
202025-12-18T04:05:09.561ZINFOcrucible-agent (worker): ok!
212025-12-18T04:05:09.579ZINFOcrucible-agent (worker): SMF ok!
222025-12-18T04:05:09.609ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50069 req_id = 7801d6ba-5231-458f-895d-72abcaab03b8 response_code = 200 uri = /crucible/0/regions
232025-12-18T04:05:11.172ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:50069 req_id = 287ff658-81d4-4aa5-97ec-c84013faba35 response_code = 200 uri = /crucible/0/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4
242025-12-18T04:05:11.175ZINFOcrucible-agent (datafile): path is "/data/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4"
252025-12-18T04:05:11.182ZINFOcrucible-agent (datafile): dataset is oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4
262025-12-18T04:05:11.188ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4@18949182-d4ea-490e-a3e1-019d40611313
272025-12-18T04:05:11.188ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4", "18949182-d4ea-490e-a3e1-019d40611313"]
282025-12-18T04:05:11.197ZINFOcrucible-agent (dropshot): request completed latency_us = 21391 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:50069 req_id = e3f9bcfb-78ec-4c98-8971-ec8ee71ab344 response_code = 200 uri = /crucible/0/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4/snapshots/18949182-d4ea-490e-a3e1-019d40611313
292025-12-18T04:05:11.201ZINFOcrucible-agent (datafile): path is "/data/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4"
302025-12-18T04:05:11.206ZINFOcrucible-agent (datafile): dataset is oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4
312025-12-18T04:05:11.212ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4@18949182-d4ea-490e-a3e1-019d40611313
322025-12-18T04:05:11.212ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4", "18949182-d4ea-490e-a3e1-019d40611313"]
332025-12-18T04:05:11.219ZINFOcrucible-agent (datafile): requesting running snapshot fb56aca2-2f24-462f-8d34-13ced0bfb6e4-18949182-d4ea-490e-a3e1-019d40611313 state: Requested
342025-12-18T04:05:11.220ZINFOcrucible-agent (dropshot): request completed latency_us = 18845 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50069 req_id = d7175915-e662-41c6-964a-d254ac348d3a response_code = 200 uri = /crucible/0/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4/snapshots/18949182-d4ea-490e-a3e1-019d40611313/run
352025-12-18T04:05:11.220ZINFOcrucible-agent (worker): applying SMF actions for region fb56aca2-2f24-462f-8d34-13ced0bfb6e4 running snapshot 18949182-d4ea-490e-a3e1-019d40611313 (state Requested)...
362025-12-18T04:05:11.224ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-fb56aca2-2f24-462f-8d34-13ced0bfb6e4-18949182-d4ea-490e-a3e1-019d40611313
372025-12-18T04:05:11.232ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-fb56aca2-2f24-462f-8d34-13ced0bfb6e4-18949182-d4ea-490e-a3e1-019d40611313
382025-12-18T04:05:11.241ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-fb56aca2-2f24-462f-8d34-13ced0bfb6e4-18949182-d4ea-490e-a3e1-019d40611313
392025-12-18T04:05:11.241ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4/.zfs/snapshot/18949182-d4ea-490e-a3e1-019d40611313
402025-12-18T04:05:11.242ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-12-18T04:05:11.242ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-12-18T04:05:11.242ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432025-12-18T04:05:11.242ZINFOcrucible-agent (worker): commit
442025-12-18T04:05:11.245ZINFOcrucible-agent (worker): ok!
452025-12-18T04:05:11.249ZINFOcrucible-agent (worker): SMF ok!
462025-12-18T04:05:11.249ZINFOcrucible-agent (datafile): region fb56aca2-2f24-462f-8d34-13ced0bfb6e4 running snapshot 18949182-d4ea-490e-a3e1-019d40611313 state: Requested -> Created
472025-12-18T04:05:11.453ZINFOcrucible-agent (datafile): path is "/data/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4"
482025-12-18T04:05:11.460ZINFOcrucible-agent (datafile): dataset is oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4
492025-12-18T04:05:11.467ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4@18949182-d4ea-490e-a3e1-019d40611313
502025-12-18T04:05:11.467ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_91531a49-d500-42cd-836c-4e70a5ac02bd/crucible/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4", "18949182-d4ea-490e-a3e1-019d40611313"]
512025-12-18T04:05:11.473ZINFOcrucible-agent (dropshot): request completed latency_us = 19924 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50069 req_id = 5b819ac4-8674-4be4-83d5-e2d865ad10bd response_code = 200 uri = /crucible/0/regions/fb56aca2-2f24-462f-8d34-13ced0bfb6e4/snapshots/18949182-d4ea-490e-a3e1-019d40611313/run