Max level shown:
12026-05-16T01:09:43.213ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::a]:53150
22026-05-16T01:09:43.213ZINFOcrucible-agent (datafile): region 8ac18eb7-1a8a-406c-a850-8cb77d85ec78 state: Requested
32026-05-16T01:09:43.214ZINFOcrucible-agent (dropshot): request completed latency_us = 744 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:53150 req_id = 666a5925-5f43-4720-943f-778752722d98 response_code = 200 uri = /crucible/0/regions
42026-05-16T01:09:43.214ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-05-16T01:09:43.221ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/8ac18eb7-1a8a-406c-a850-8cb77d85ec78
62026-05-16T01:09:43.221ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/8ac18eb7-1a8a-406c-a850-8cb77d85ec78
72026-05-16T01:09:43.264ZINFOcrucible-agent (worker): creating region Region { id: RegionId("8ac18eb7-1a8a-406c-a850-8cb77d85ec78"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/8ac18eb7-1a8a-406c-a850-8cb77d85ec78" region = 8ac18eb7-1a8a-406c-a850-8cb77d85ec78
82026-05-16T01:09:43.264ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "8ac18eb7-1a8a-406c-a850-8cb77d85ec78" "--data" "/data/regions/8ac18eb7-1a8a-406c-a850-8cb77d85ec78" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 8ac18eb7-1a8a-406c-a850-8cb77d85ec78
92026-05-16T01:09:43.318ZINFOcrucible-agent (worker): region files created ok region = 8ac18eb7-1a8a-406c-a850-8cb77d85ec78
102026-05-16T01:09:43.318ZINFOcrucible-agent (datafile): region 8ac18eb7-1a8a-406c-a850-8cb77d85ec78 state: Requested -> Created
112026-05-16T01:09:43.319ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-05-16T01:09:43.319ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-8ac18eb7-1a8a-406c-a850-8cb77d85ec78
132026-05-16T01:09:43.322ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-8ac18eb7-1a8a-406c-a850-8cb77d85ec78
142026-05-16T01:09:43.337ZINFOcrucible-agent (worker): creating config property group
152026-05-16T01:09:43.338ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8ac18eb7-1a8a-406c-a850-8cb77d85ec78
162026-05-16T01:09:43.338ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8ac18eb7-1a8a-406c-a850-8cb77d85ec78
172026-05-16T01:09:43.338ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-05-16T01:09:43.338ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
192026-05-16T01:09:43.339ZINFOcrucible-agent (worker): commit
202026-05-16T01:09:43.340ZINFOcrucible-agent (worker): ok!
212026-05-16T01:09:43.351ZINFOcrucible-agent (worker): SMF ok!
222026-05-16T01:09:43.443ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:53150 req_id = fad3bcb0-b1d6-4a57-be04-901aae8c9e65 response_code = 200 uri = /crucible/0/regions
232026-05-16T01:09:45.032ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:53150 req_id = b635ea16-dd46-4b04-930e-7b6697c4d974 response_code = 200 uri = /crucible/0/regions/f3306782-43f5-4a19-b27a-6c3a55b12141
242026-05-16T01:09:45.035ZINFOcrucible-agent (datafile): path is "/data/regions/f3306782-43f5-4a19-b27a-6c3a55b12141"
252026-05-16T01:09:45.043ZINFOcrucible-agent (datafile): dataset is oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/f3306782-43f5-4a19-b27a-6c3a55b12141
262026-05-16T01:09:45.051ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/f3306782-43f5-4a19-b27a-6c3a55b12141@62389c1f-2373-483c-b6fa-2dab47d00ae0
272026-05-16T01:09:45.051ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/f3306782-43f5-4a19-b27a-6c3a55b12141", "62389c1f-2373-483c-b6fa-2dab47d00ae0"]
282026-05-16T01:09:45.062ZINFOcrucible-agent (dropshot): request completed latency_us = 26575 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:53150 req_id = 3c7c7a12-a488-4147-896d-febc1186e729 response_code = 200 uri = /crucible/0/regions/f3306782-43f5-4a19-b27a-6c3a55b12141/snapshots/62389c1f-2373-483c-b6fa-2dab47d00ae0
292026-05-16T01:09:45.066ZINFOcrucible-agent (datafile): path is "/data/regions/f3306782-43f5-4a19-b27a-6c3a55b12141"
302026-05-16T01:09:45.074ZINFOcrucible-agent (datafile): dataset is oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/f3306782-43f5-4a19-b27a-6c3a55b12141
312026-05-16T01:09:45.081ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/f3306782-43f5-4a19-b27a-6c3a55b12141@62389c1f-2373-483c-b6fa-2dab47d00ae0
322026-05-16T01:09:45.081ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/f3306782-43f5-4a19-b27a-6c3a55b12141", "62389c1f-2373-483c-b6fa-2dab47d00ae0"]
332026-05-16T01:09:45.091ZINFOcrucible-agent (datafile): requesting running snapshot f3306782-43f5-4a19-b27a-6c3a55b12141-62389c1f-2373-483c-b6fa-2dab47d00ae0 state: Requested
342026-05-16T01:09:45.091ZINFOcrucible-agent (worker): applying SMF actions for region f3306782-43f5-4a19-b27a-6c3a55b12141 running snapshot 62389c1f-2373-483c-b6fa-2dab47d00ae0 (state Requested)...
352026-05-16T01:09:45.091ZINFOcrucible-agent (dropshot): request completed latency_us = 25250 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:53150 req_id = c8e5892c-ea02-4b00-8d82-46c837b1b666 response_code = 200 uri = /crucible/0/regions/f3306782-43f5-4a19-b27a-6c3a55b12141/snapshots/62389c1f-2373-483c-b6fa-2dab47d00ae0/run
362026-05-16T01:09:45.096ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-f3306782-43f5-4a19-b27a-6c3a55b12141-62389c1f-2373-483c-b6fa-2dab47d00ae0
372026-05-16T01:09:45.100ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-f3306782-43f5-4a19-b27a-6c3a55b12141-62389c1f-2373-483c-b6fa-2dab47d00ae0
382026-05-16T01:09:45.106ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-f3306782-43f5-4a19-b27a-6c3a55b12141-62389c1f-2373-483c-b6fa-2dab47d00ae0
392026-05-16T01:09:45.106ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f3306782-43f5-4a19-b27a-6c3a55b12141/.zfs/snapshot/62389c1f-2373-483c-b6fa-2dab47d00ae0
402026-05-16T01:09:45.106ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-05-16T01:09:45.106ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-05-16T01:09:45.106ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
432026-05-16T01:09:45.106ZINFOcrucible-agent (worker): commit
442026-05-16T01:09:45.108ZINFOcrucible-agent (worker): ok!
452026-05-16T01:09:45.111ZINFOcrucible-agent (worker): SMF ok!
462026-05-16T01:09:45.111ZINFOcrucible-agent (datafile): region f3306782-43f5-4a19-b27a-6c3a55b12141 running snapshot 62389c1f-2373-483c-b6fa-2dab47d00ae0 state: Requested -> Created
472026-05-16T01:09:45.462ZINFOcrucible-agent (datafile): path is "/data/regions/f3306782-43f5-4a19-b27a-6c3a55b12141"
482026-05-16T01:09:45.469ZINFOcrucible-agent (datafile): dataset is oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/f3306782-43f5-4a19-b27a-6c3a55b12141
492026-05-16T01:09:45.474ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/f3306782-43f5-4a19-b27a-6c3a55b12141@62389c1f-2373-483c-b6fa-2dab47d00ae0
502026-05-16T01:09:45.474ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_380b92fd-19d9-4b1c-bb33-c29c81e2da2f/crucible/regions/f3306782-43f5-4a19-b27a-6c3a55b12141", "62389c1f-2373-483c-b6fa-2dab47d00ae0"]
512026-05-16T01:09:45.481ZINFOcrucible-agent (dropshot): request completed latency_us = 19378 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:53150 req_id = 626887fd-4982-42e5-93d9-fec41b6cbd21 response_code = 200 uri = /crucible/0/regions/f3306782-43f5-4a19-b27a-6c3a55b12141/snapshots/62389c1f-2373-483c-b6fa-2dab47d00ae0/run