Max level shown:
12025-02-21T07:36:56.932ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::a]:39088
22025-02-21T07:36:56.932ZINFOcrucible-agent (datafile): region db8f6932-af73-4cfc-ab59-dcd129a530d2 state: Requested
32025-02-21T07:36:56.933ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-02-21T07:36:56.933ZINFOcrucible-agent (dropshot): request completed latency_us = 1163 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:39088 req_id = 3388dbc8-4ed4-4075-856f-cb329a4c453e response_code = 200 uri = /crucible/0/regions
52025-02-21T07:36:56.944ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/db8f6932-af73-4cfc-ab59-dcd129a530d2
62025-02-21T07:36:56.944ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/db8f6932-af73-4cfc-ab59-dcd129a530d2
72025-02-21T07:36:57.031ZINFOcrucible-agent (worker): creating region Region { id: RegionId("db8f6932-af73-4cfc-ab59-dcd129a530d2"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/db8f6932-af73-4cfc-ab59-dcd129a530d2" region = db8f6932-af73-4cfc-ab59-dcd129a530d2
82025-02-21T07:36:57.031ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "db8f6932-af73-4cfc-ab59-dcd129a530d2" "--data" "/data/regions/db8f6932-af73-4cfc-ab59-dcd129a530d2" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = db8f6932-af73-4cfc-ab59-dcd129a530d2
92025-02-21T07:36:57.106ZINFOcrucible-agent (worker): region files created ok region = db8f6932-af73-4cfc-ab59-dcd129a530d2
102025-02-21T07:36:57.106ZINFOcrucible-agent (datafile): region db8f6932-af73-4cfc-ab59-dcd129a530d2 state: Requested -> Created
112025-02-21T07:36:57.106ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-02-21T07:36:57.116ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-db8f6932-af73-4cfc-ab59-dcd129a530d2
132025-02-21T07:36:57.127ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-db8f6932-af73-4cfc-ab59-dcd129a530d2
142025-02-21T07:36:57.137ZINFOcrucible-agent (worker): creating config property group
152025-02-21T07:36:57.140ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-db8f6932-af73-4cfc-ab59-dcd129a530d2
162025-02-21T07:36:57.141ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/db8f6932-af73-4cfc-ab59-dcd129a530d2
172025-02-21T07:36:57.141ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-02-21T07:36:57.141ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
192025-02-21T07:36:57.141ZINFOcrucible-agent (worker): commit
202025-02-21T07:36:57.146ZINFOcrucible-agent (worker): ok!
212025-02-21T07:36:57.149ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:39088 req_id = 079adc11-4e44-449e-aa10-56d49f08da1b response_code = 200 uri = /crucible/0/regions
222025-02-21T07:36:57.152ZINFOcrucible-agent (worker): SMF ok!
232025-02-21T07:36:58.208ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:39088 req_id = e601ee38-6b54-488d-9753-e0091fd86a98 response_code = 200 uri = /crucible/0/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34
242025-02-21T07:36:58.211ZINFOcrucible-agent (datafile): path is "/data/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34"
252025-02-21T07:36:58.221ZINFOcrucible-agent (datafile): dataset is oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34
262025-02-21T07:36:58.230ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34@bbdfbc53-562c-436f-8909-18e5b68befdc
272025-02-21T07:36:58.230ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34", "bbdfbc53-562c-436f-8909-18e5b68befdc"]
282025-02-21T07:36:58.242ZINFOcrucible-agent (dropshot): request completed latency_us = 30900 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:39088 req_id = 17e65f4c-f1e9-4540-aa27-c00090435d39 response_code = 200 uri = /crucible/0/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34/snapshots/bbdfbc53-562c-436f-8909-18e5b68befdc
292025-02-21T07:36:58.246ZINFOcrucible-agent (datafile): path is "/data/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34"
302025-02-21T07:36:58.254ZINFOcrucible-agent (datafile): dataset is oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34
312025-02-21T07:36:58.262ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34@bbdfbc53-562c-436f-8909-18e5b68befdc
322025-02-21T07:36:58.262ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34", "bbdfbc53-562c-436f-8909-18e5b68befdc"]
332025-02-21T07:36:58.271ZINFOcrucible-agent (datafile): requesting running snapshot be4c4d2b-527b-4721-b92b-6c286aff9f34-bbdfbc53-562c-436f-8909-18e5b68befdc state: Requested
342025-02-21T07:36:58.271ZINFOcrucible-agent (worker): applying SMF actions for region be4c4d2b-527b-4721-b92b-6c286aff9f34 running snapshot bbdfbc53-562c-436f-8909-18e5b68befdc (state Requested)...
352025-02-21T07:36:58.271ZINFOcrucible-agent (dropshot): request completed latency_us = 24974 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:39088 req_id = d36b19d1-8149-4adc-9ff0-cc9846c4b8c0 response_code = 200 uri = /crucible/0/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34/snapshots/bbdfbc53-562c-436f-8909-18e5b68befdc/run
362025-02-21T07:36:58.274ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-be4c4d2b-527b-4721-b92b-6c286aff9f34-bbdfbc53-562c-436f-8909-18e5b68befdc
372025-02-21T07:36:58.276ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-be4c4d2b-527b-4721-b92b-6c286aff9f34-bbdfbc53-562c-436f-8909-18e5b68befdc
382025-02-21T07:36:58.286ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-be4c4d2b-527b-4721-b92b-6c286aff9f34-bbdfbc53-562c-436f-8909-18e5b68befdc
392025-02-21T07:36:58.287ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34/.zfs/snapshot/bbdfbc53-562c-436f-8909-18e5b68befdc
402025-02-21T07:36:58.287ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-02-21T07:36:58.287ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-02-21T07:36:58.287ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
432025-02-21T07:36:58.287ZINFOcrucible-agent (worker): commit
442025-02-21T07:36:58.290ZINFOcrucible-agent (worker): ok!
452025-02-21T07:36:58.295ZINFOcrucible-agent (worker): SMF ok!
462025-02-21T07:36:58.295ZINFOcrucible-agent (datafile): region be4c4d2b-527b-4721-b92b-6c286aff9f34 running snapshot bbdfbc53-562c-436f-8909-18e5b68befdc state: Requested -> Created
472025-02-21T07:36:58.493ZINFOcrucible-agent (datafile): path is "/data/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34"
482025-02-21T07:36:58.501ZINFOcrucible-agent (datafile): dataset is oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34
492025-02-21T07:36:58.510ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34@bbdfbc53-562c-436f-8909-18e5b68befdc
502025-02-21T07:36:58.510ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_592f85d2-9ce2-4a02-89cc-fa9c7cf537b1/crucible/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34", "bbdfbc53-562c-436f-8909-18e5b68befdc"]
512025-02-21T07:36:58.518ZINFOcrucible-agent (dropshot): request completed latency_us = 25672 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:39088 req_id = 49946538-f86b-44be-bafd-f0aebacd5e07 response_code = 200 uri = /crucible/0/regions/be4c4d2b-527b-4721-b92b-6c286aff9f34/snapshots/bbdfbc53-562c-436f-8909-18e5b68befdc/run