Max level shown:
12026-03-20T20:06:06.578ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::b]:53499
22026-03-20T20:06:06.578ZINFOcrucible-agent (datafile): region 89643e8d-4f52-4c4d-8b9c-f54d073ce78b state: Requested
32026-03-20T20:06:06.578ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-03-20T20:06:06.578ZINFOcrucible-agent (dropshot): request completed latency_us = 571 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:53499 req_id = 895c9a0a-8733-4849-a776-b0c3329df15f response_code = 200 uri = /crucible/0/regions
52026-03-20T20:06:06.586ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/89643e8d-4f52-4c4d-8b9c-f54d073ce78b
62026-03-20T20:06:06.586ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/89643e8d-4f52-4c4d-8b9c-f54d073ce78b
72026-03-20T20:06:06.636ZINFOcrucible-agent (worker): creating region Region { id: RegionId("89643e8d-4f52-4c4d-8b9c-f54d073ce78b"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/89643e8d-4f52-4c4d-8b9c-f54d073ce78b" region = 89643e8d-4f52-4c4d-8b9c-f54d073ce78b
82026-03-20T20:06:06.636ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "89643e8d-4f52-4c4d-8b9c-f54d073ce78b" "--data" "/data/regions/89643e8d-4f52-4c4d-8b9c-f54d073ce78b" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 89643e8d-4f52-4c4d-8b9c-f54d073ce78b
92026-03-20T20:06:06.716ZINFOcrucible-agent (worker): region files created ok region = 89643e8d-4f52-4c4d-8b9c-f54d073ce78b
102026-03-20T20:06:06.716ZINFOcrucible-agent (datafile): region 89643e8d-4f52-4c4d-8b9c-f54d073ce78b state: Requested -> Created
112026-03-20T20:06:06.716ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-03-20T20:06:06.716ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-89643e8d-4f52-4c4d-8b9c-f54d073ce78b
132026-03-20T20:06:06.725ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:53499 req_id = f4f826ca-8a34-4452-ab26-d5beee64c8a7 response_code = 200 uri = /crucible/0/regions
142026-03-20T20:06:06.726ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-89643e8d-4f52-4c4d-8b9c-f54d073ce78b
152026-03-20T20:06:06.748ZINFOcrucible-agent (worker): creating config property group
162026-03-20T20:06:06.751ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-89643e8d-4f52-4c4d-8b9c-f54d073ce78b
172026-03-20T20:06:06.752ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/89643e8d-4f52-4c4d-8b9c-f54d073ce78b
182026-03-20T20:06:06.752ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192026-03-20T20:06:06.752ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
202026-03-20T20:06:06.752ZINFOcrucible-agent (worker): commit
212026-03-20T20:06:06.756ZINFOcrucible-agent (worker): ok!
222026-03-20T20:06:06.770ZINFOcrucible-agent (worker): SMF ok!
232026-03-20T20:06:08.513ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:53499 req_id = 1704f013-fc58-4d77-8f3a-b147a6de632c response_code = 200 uri = /crucible/0/regions/9b50817f-cbad-4d08-a019-f208f93edac5
242026-03-20T20:06:08.516ZINFOcrucible-agent (datafile): path is "/data/regions/9b50817f-cbad-4d08-a019-f208f93edac5"
252026-03-20T20:06:08.522ZINFOcrucible-agent (datafile): dataset is oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/9b50817f-cbad-4d08-a019-f208f93edac5
262026-03-20T20:06:08.528ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/9b50817f-cbad-4d08-a019-f208f93edac5@d26619b6-5322-4641-9f77-b1362ace944d
272026-03-20T20:06:08.528ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/9b50817f-cbad-4d08-a019-f208f93edac5", "d26619b6-5322-4641-9f77-b1362ace944d"]
282026-03-20T20:06:08.537ZINFOcrucible-agent (dropshot): request completed latency_us = 20399 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:53499 req_id = b550867c-c73c-44d5-88c6-85079d64ee24 response_code = 200 uri = /crucible/0/regions/9b50817f-cbad-4d08-a019-f208f93edac5/snapshots/d26619b6-5322-4641-9f77-b1362ace944d
292026-03-20T20:06:08.540ZINFOcrucible-agent (datafile): path is "/data/regions/9b50817f-cbad-4d08-a019-f208f93edac5"
302026-03-20T20:06:08.546ZINFOcrucible-agent (datafile): dataset is oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/9b50817f-cbad-4d08-a019-f208f93edac5
312026-03-20T20:06:08.552ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/9b50817f-cbad-4d08-a019-f208f93edac5@d26619b6-5322-4641-9f77-b1362ace944d
322026-03-20T20:06:08.552ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/9b50817f-cbad-4d08-a019-f208f93edac5", "d26619b6-5322-4641-9f77-b1362ace944d"]
332026-03-20T20:06:08.558ZINFOcrucible-agent (datafile): requesting running snapshot 9b50817f-cbad-4d08-a019-f208f93edac5-d26619b6-5322-4641-9f77-b1362ace944d state: Requested
342026-03-20T20:06:08.558ZINFOcrucible-agent (worker): applying SMF actions for region 9b50817f-cbad-4d08-a019-f208f93edac5 running snapshot d26619b6-5322-4641-9f77-b1362ace944d (state Requested)...
352026-03-20T20:06:08.558ZINFOcrucible-agent (dropshot): request completed latency_us = 18178 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:53499 req_id = da778210-0809-4d83-bfa2-135cd6cd0b7d response_code = 200 uri = /crucible/0/regions/9b50817f-cbad-4d08-a019-f208f93edac5/snapshots/d26619b6-5322-4641-9f77-b1362ace944d/run
362026-03-20T20:06:08.563ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-9b50817f-cbad-4d08-a019-f208f93edac5-d26619b6-5322-4641-9f77-b1362ace944d
372026-03-20T20:06:08.575ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-9b50817f-cbad-4d08-a019-f208f93edac5-d26619b6-5322-4641-9f77-b1362ace944d
382026-03-20T20:06:08.586ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-9b50817f-cbad-4d08-a019-f208f93edac5-d26619b6-5322-4641-9f77-b1362ace944d
392026-03-20T20:06:08.586ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/9b50817f-cbad-4d08-a019-f208f93edac5/.zfs/snapshot/d26619b6-5322-4641-9f77-b1362ace944d
402026-03-20T20:06:08.586ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-03-20T20:06:08.586ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-03-20T20:06:08.586ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432026-03-20T20:06:08.586ZINFOcrucible-agent (worker): commit
442026-03-20T20:06:08.590ZINFOcrucible-agent (worker): ok!
452026-03-20T20:06:08.595ZINFOcrucible-agent (worker): SMF ok!
462026-03-20T20:06:08.595ZINFOcrucible-agent (datafile): region 9b50817f-cbad-4d08-a019-f208f93edac5 running snapshot d26619b6-5322-4641-9f77-b1362ace944d state: Requested -> Created
472026-03-20T20:06:08.846ZINFOcrucible-agent (datafile): path is "/data/regions/9b50817f-cbad-4d08-a019-f208f93edac5"
482026-03-20T20:06:08.854ZINFOcrucible-agent (datafile): dataset is oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/9b50817f-cbad-4d08-a019-f208f93edac5
492026-03-20T20:06:08.861ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/9b50817f-cbad-4d08-a019-f208f93edac5@d26619b6-5322-4641-9f77-b1362ace944d
502026-03-20T20:06:08.861ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_4b664866-74d5-4696-a337-22b59e9dd248/crucible/regions/9b50817f-cbad-4d08-a019-f208f93edac5", "d26619b6-5322-4641-9f77-b1362ace944d"]
512026-03-20T20:06:08.868ZINFOcrucible-agent (dropshot): request completed latency_us = 22015 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:53499 req_id = 7aafdee0-705e-4cfd-8581-3c847ea099d1 response_code = 200 uri = /crucible/0/regions/9b50817f-cbad-4d08-a019-f208f93edac5/snapshots/d26619b6-5322-4641-9f77-b1362ace944d/run