Max level shown:
12025-11-18T00:07:52.994ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::b]:43638
22025-11-18T00:07:52.994ZINFOcrucible-agent (datafile): region 887584ac-e0c5-4da2-b758-78e94d9d979d state: Requested
32025-11-18T00:07:52.996ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-11-18T00:07:52.996ZINFOcrucible-agent (dropshot): request completed latency_us = 2162 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43638 req_id = 827f5da9-9675-44cd-83c7-0d5c077003a2 response_code = 200 uri = /crucible/0/regions
52025-11-18T00:07:53.007ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/887584ac-e0c5-4da2-b758-78e94d9d979d
62025-11-18T00:07:53.007ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/887584ac-e0c5-4da2-b758-78e94d9d979d
72025-11-18T00:07:53.088ZINFOcrucible-agent (worker): creating region Region { id: RegionId("887584ac-e0c5-4da2-b758-78e94d9d979d"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/887584ac-e0c5-4da2-b758-78e94d9d979d" region = 887584ac-e0c5-4da2-b758-78e94d9d979d
82025-11-18T00:07:53.088ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "887584ac-e0c5-4da2-b758-78e94d9d979d" "--data" "/data/regions/887584ac-e0c5-4da2-b758-78e94d9d979d" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 887584ac-e0c5-4da2-b758-78e94d9d979d
92025-11-18T00:07:53.190ZINFOcrucible-agent (dropshot): request completed latency_us = 78 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43638 req_id = aab469a9-d081-407c-baa4-23b8123c6fb2 response_code = 200 uri = /crucible/0/regions
102025-11-18T00:07:53.236ZINFOcrucible-agent (worker): region files created ok region = 887584ac-e0c5-4da2-b758-78e94d9d979d
112025-11-18T00:07:53.236ZINFOcrucible-agent (datafile): region 887584ac-e0c5-4da2-b758-78e94d9d979d state: Requested -> Created
122025-11-18T00:07:53.236ZINFOcrucible-agent (worker): applying SMF actions post create...
132025-11-18T00:07:53.256ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-887584ac-e0c5-4da2-b758-78e94d9d979d
142025-11-18T00:07:53.267ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-887584ac-e0c5-4da2-b758-78e94d9d979d
152025-11-18T00:07:53.299ZINFOcrucible-agent (worker): creating config property group
162025-11-18T00:07:53.310ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-887584ac-e0c5-4da2-b758-78e94d9d979d
172025-11-18T00:07:53.311ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/887584ac-e0c5-4da2-b758-78e94d9d979d
182025-11-18T00:07:53.311ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-11-18T00:07:53.311ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
202025-11-18T00:07:53.311ZINFOcrucible-agent (worker): commit
212025-11-18T00:07:53.318ZINFOcrucible-agent (worker): ok!
222025-11-18T00:07:53.329ZINFOcrucible-agent (worker): SMF ok!
232025-11-18T00:07:53.744ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43638 req_id = 6b3bfbf8-aa94-4f4e-a3f2-37ddb977a32f response_code = 200 uri = /crucible/0/regions
242025-11-18T00:07:54.874ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43638 req_id = 11e2690f-5e58-4be4-9e92-1b28d2ff42f8 response_code = 200 uri = /crucible/0/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821
252025-11-18T00:07:54.877ZINFOcrucible-agent (datafile): path is "/data/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821"
262025-11-18T00:07:54.883ZINFOcrucible-agent (datafile): dataset is oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821
272025-11-18T00:07:54.890ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821@302e67d8-0457-4826-8058-830e75ae370a
282025-11-18T00:07:54.890ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821", "302e67d8-0457-4826-8058-830e75ae370a"]
292025-11-18T00:07:54.900ZINFOcrucible-agent (dropshot): request completed latency_us = 23702 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43638 req_id = f40a669c-5459-42ba-965f-252575aab0d2 response_code = 200 uri = /crucible/0/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821/snapshots/302e67d8-0457-4826-8058-830e75ae370a
302025-11-18T00:07:54.905ZINFOcrucible-agent (datafile): path is "/data/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821"
312025-11-18T00:07:54.912ZINFOcrucible-agent (datafile): dataset is oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821
322025-11-18T00:07:54.918ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821@302e67d8-0457-4826-8058-830e75ae370a
332025-11-18T00:07:54.918ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821", "302e67d8-0457-4826-8058-830e75ae370a"]
342025-11-18T00:07:54.929ZINFOcrucible-agent (datafile): requesting running snapshot 2a348e0f-7899-47a6-a06b-5b6d54c31821-302e67d8-0457-4826-8058-830e75ae370a state: Requested
352025-11-18T00:07:54.929ZINFOcrucible-agent (dropshot): request completed latency_us = 24845 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43638 req_id = 2a346baa-ece2-411c-bec1-236b8ccaa735 response_code = 200 uri = /crucible/0/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821/snapshots/302e67d8-0457-4826-8058-830e75ae370a/run
362025-11-18T00:07:54.929ZINFOcrucible-agent (worker): applying SMF actions for region 2a348e0f-7899-47a6-a06b-5b6d54c31821 running snapshot 302e67d8-0457-4826-8058-830e75ae370a (state Requested)...
372025-11-18T00:07:54.935ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-2a348e0f-7899-47a6-a06b-5b6d54c31821-302e67d8-0457-4826-8058-830e75ae370a
382025-11-18T00:07:54.952ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-2a348e0f-7899-47a6-a06b-5b6d54c31821-302e67d8-0457-4826-8058-830e75ae370a
392025-11-18T00:07:54.979ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-2a348e0f-7899-47a6-a06b-5b6d54c31821-302e67d8-0457-4826-8058-830e75ae370a
402025-11-18T00:07:54.979ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821/.zfs/snapshot/302e67d8-0457-4826-8058-830e75ae370a
412025-11-18T00:07:54.979ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
422025-11-18T00:07:54.979ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
432025-11-18T00:07:54.979ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
442025-11-18T00:07:54.979ZINFOcrucible-agent (worker): commit
452025-11-18T00:07:54.987ZINFOcrucible-agent (worker): ok!
462025-11-18T00:07:55.000ZINFOcrucible-agent (worker): SMF ok!
472025-11-18T00:07:55.000ZINFOcrucible-agent (datafile): region 2a348e0f-7899-47a6-a06b-5b6d54c31821 running snapshot 302e67d8-0457-4826-8058-830e75ae370a state: Requested -> Created
482025-11-18T00:07:55.151ZINFOcrucible-agent (datafile): path is "/data/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821"
492025-11-18T00:07:55.158ZINFOcrucible-agent (datafile): dataset is oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821
502025-11-18T00:07:55.164ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821@302e67d8-0457-4826-8058-830e75ae370a
512025-11-18T00:07:55.164ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_32493221-44e2-431a-a23d-ddfabd42f8e9/crucible/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821", "302e67d8-0457-4826-8058-830e75ae370a"]
522025-11-18T00:07:55.171ZINFOcrucible-agent (dropshot): request completed latency_us = 19263 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43638 req_id = 5e2e7cbe-6f6e-437d-a9a8-8d10e8774ead response_code = 200 uri = /crucible/0/regions/2a348e0f-7899-47a6-a06b-5b6d54c31821/snapshots/302e67d8-0457-4826-8058-830e75ae370a/run