Max level shown:
12025-02-25T04:33:21.918ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::c]:36700
22025-02-25T04:33:21.919ZINFOcrucible-agent (datafile): region d5916500-62cb-4804-9f73-1a8798da15fa state: Requested
32025-02-25T04:33:21.923ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-02-25T04:33:21.923ZINFOcrucible-agent (dropshot): request completed latency_us = 4915 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:36700 req_id = 97de0193-cf5c-482c-85a0-9ba0d5b64e86 response_code = 200 uri = /crucible/0/regions
52025-02-25T04:33:21.938ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/d5916500-62cb-4804-9f73-1a8798da15fa
62025-02-25T04:33:21.938ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/d5916500-62cb-4804-9f73-1a8798da15fa
72025-02-25T04:33:22.165ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:36700 req_id = e32c06e4-6d9a-43a7-bb74-24ec661c35e5 response_code = 200 uri = /crucible/0/regions
82025-02-25T04:33:22.232ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d5916500-62cb-4804-9f73-1a8798da15fa"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d5916500-62cb-4804-9f73-1a8798da15fa" region = d5916500-62cb-4804-9f73-1a8798da15fa
92025-02-25T04:33:22.232ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d5916500-62cb-4804-9f73-1a8798da15fa" "--data" "/data/regions/d5916500-62cb-4804-9f73-1a8798da15fa" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = d5916500-62cb-4804-9f73-1a8798da15fa
102025-02-25T04:33:22.760ZINFOcrucible-agent (dropshot): request completed latency_us = 102 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:36700 req_id = d9d445b6-875e-4e5d-aaa1-f73b83f0204e response_code = 200 uri = /crucible/0/regions
112025-02-25T04:33:22.830ZINFOcrucible-agent (worker): region files created ok region = d5916500-62cb-4804-9f73-1a8798da15fa
122025-02-25T04:33:22.830ZINFOcrucible-agent (datafile): region d5916500-62cb-4804-9f73-1a8798da15fa state: Requested -> Created
132025-02-25T04:33:22.830ZINFOcrucible-agent (worker): applying SMF actions post create...
142025-02-25T04:33:22.838ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d5916500-62cb-4804-9f73-1a8798da15fa
152025-02-25T04:33:22.889ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d5916500-62cb-4804-9f73-1a8798da15fa
162025-02-25T04:33:22.942ZINFOcrucible-agent (worker): creating config property group
172025-02-25T04:33:22.954ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d5916500-62cb-4804-9f73-1a8798da15fa
182025-02-25T04:33:22.954ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d5916500-62cb-4804-9f73-1a8798da15fa
192025-02-25T04:33:22.954ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
202025-02-25T04:33:22.954ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
212025-02-25T04:33:22.954ZINFOcrucible-agent (worker): commit
222025-02-25T04:33:22.987ZINFOcrucible-agent (worker): ok!
232025-02-25T04:33:23.022ZINFOcrucible-agent (worker): SMF ok!
242025-02-25T04:33:23.736ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::b]:49238
252025-02-25T04:33:23.737ZINFOcrucible-agent (dropshot): request completed latency_us = 41 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:49238 req_id = 27d35682-632c-4496-883f-6aaa23b39403 response_code = 200 uri = /crucible/0/regions/d5916500-62cb-4804-9f73-1a8798da15fa
262025-02-25T04:33:24.074ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:36700 req_id = f4a8f579-2915-4c53-bdf3-d97c01d33114 response_code = 200 uri = /crucible/0/regions
272025-02-25T04:33:26.142ZINFOcrucible-agent (dropshot): request completed latency_us = 44 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:36700 req_id = 880a1c8c-7c85-4f16-a0cd-63711cc8b459 response_code = 200 uri = /crucible/0/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e
282025-02-25T04:33:26.146ZINFOcrucible-agent (datafile): path is "/data/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e"
292025-02-25T04:33:26.155ZINFOcrucible-agent (datafile): dataset is oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e
302025-02-25T04:33:26.164ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e@3b576f69-edfa-432f-b1cc-0c4cd89de405
312025-02-25T04:33:26.164ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e", "3b576f69-edfa-432f-b1cc-0c4cd89de405"]
322025-02-25T04:33:26.176ZINFOcrucible-agent (dropshot): request completed latency_us = 29626 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:36700 req_id = 7b161d56-13a1-46a9-ac82-a31e9a58d5fa response_code = 200 uri = /crucible/0/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e/snapshots/3b576f69-edfa-432f-b1cc-0c4cd89de405
332025-02-25T04:33:26.179ZINFOcrucible-agent (datafile): path is "/data/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e"
342025-02-25T04:33:26.187ZINFOcrucible-agent (datafile): dataset is oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e
352025-02-25T04:33:26.195ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e@3b576f69-edfa-432f-b1cc-0c4cd89de405
362025-02-25T04:33:26.195ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e", "3b576f69-edfa-432f-b1cc-0c4cd89de405"]
372025-02-25T04:33:26.203ZINFOcrucible-agent (datafile): requesting running snapshot 3ceee25e-5b78-4e92-ab0d-9b458bfded6e-3b576f69-edfa-432f-b1cc-0c4cd89de405 state: Requested
382025-02-25T04:33:26.203ZINFOcrucible-agent (worker): applying SMF actions for region 3ceee25e-5b78-4e92-ab0d-9b458bfded6e running snapshot 3b576f69-edfa-432f-b1cc-0c4cd89de405 (state Requested)...
392025-02-25T04:33:26.203ZINFOcrucible-agent (dropshot): request completed latency_us = 24533 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:36700 req_id = ecf4b797-5593-40c2-84f2-8dfadae00ba1 response_code = 200 uri = /crucible/0/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e/snapshots/3b576f69-edfa-432f-b1cc-0c4cd89de405/run
402025-02-25T04:33:26.206ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-3ceee25e-5b78-4e92-ab0d-9b458bfded6e-3b576f69-edfa-432f-b1cc-0c4cd89de405
412025-02-25T04:33:26.211ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-3ceee25e-5b78-4e92-ab0d-9b458bfded6e-3b576f69-edfa-432f-b1cc-0c4cd89de405
422025-02-25T04:33:26.279ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-3ceee25e-5b78-4e92-ab0d-9b458bfded6e-3b576f69-edfa-432f-b1cc-0c4cd89de405
432025-02-25T04:33:26.280ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e/.zfs/snapshot/3b576f69-edfa-432f-b1cc-0c4cd89de405
442025-02-25T04:33:26.280ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
452025-02-25T04:33:26.280ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
462025-02-25T04:33:26.280ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
472025-02-25T04:33:26.280ZINFOcrucible-agent (worker): commit
482025-02-25T04:33:26.294ZINFOcrucible-agent (worker): ok!
492025-02-25T04:33:26.315ZINFOcrucible-agent (worker): SMF ok!
502025-02-25T04:33:26.315ZINFOcrucible-agent (datafile): region 3ceee25e-5b78-4e92-ab0d-9b458bfded6e running snapshot 3b576f69-edfa-432f-b1cc-0c4cd89de405 state: Requested -> Created
512025-02-25T04:33:26.337ZINFOcrucible-agent (datafile): path is "/data/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e"
522025-02-25T04:33:26.347ZINFOcrucible-agent (datafile): dataset is oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e
532025-02-25T04:33:26.356ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e@3b576f69-edfa-432f-b1cc-0c4cd89de405
542025-02-25T04:33:26.356ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_dc8b73cb-89ef-41a6-8fed-906219a5be24/crucible/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e", "3b576f69-edfa-432f-b1cc-0c4cd89de405"]
552025-02-25T04:33:26.364ZINFOcrucible-agent (dropshot): request completed latency_us = 26450 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:36700 req_id = 9b332232-ef7f-4e02-bbb8-e7eb6c623c36 response_code = 200 uri = /crucible/0/regions/3ceee25e-5b78-4e92-ab0d-9b458bfded6e/snapshots/3b576f69-edfa-432f-b1cc-0c4cd89de405/run