Max level shown:
12025-08-28T22:03:02.253ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:62780
22025-08-28T22:03:02.253ZINFOcrucible-agent (datafile): region e1c591a2-e77b-4550-9466-5dd2bf95c190 state: Requested
32025-08-28T22:03:02.254ZINFOcrucible-agent (dropshot): request completed latency_us = 632 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62780 req_id = abeb6fdd-d7e3-47a9-a302-a0d7b2c8b4f4 response_code = 200 uri = /crucible/0/regions
42025-08-28T22:03:02.254ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52025-08-28T22:03:02.268ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/e1c591a2-e77b-4550-9466-5dd2bf95c190
62025-08-28T22:03:02.268ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/e1c591a2-e77b-4550-9466-5dd2bf95c190
72025-08-28T22:03:02.333ZINFOcrucible-agent (worker): creating region Region { id: RegionId("e1c591a2-e77b-4550-9466-5dd2bf95c190"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/e1c591a2-e77b-4550-9466-5dd2bf95c190" region = e1c591a2-e77b-4550-9466-5dd2bf95c190
82025-08-28T22:03:02.334ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "e1c591a2-e77b-4550-9466-5dd2bf95c190" "--data" "/data/regions/e1c591a2-e77b-4550-9466-5dd2bf95c190" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = e1c591a2-e77b-4550-9466-5dd2bf95c190
92025-08-28T22:03:02.417ZINFOcrucible-agent (worker): region files created ok region = e1c591a2-e77b-4550-9466-5dd2bf95c190
102025-08-28T22:03:02.417ZINFOcrucible-agent (datafile): region e1c591a2-e77b-4550-9466-5dd2bf95c190 state: Requested -> Created
112025-08-28T22:03:02.417ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-08-28T22:03:02.419ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-e1c591a2-e77b-4550-9466-5dd2bf95c190
132025-08-28T22:03:02.442ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-e1c591a2-e77b-4550-9466-5dd2bf95c190
142025-08-28T22:03:02.477ZINFOcrucible-agent (worker): creating config property group
152025-08-28T22:03:02.484ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-e1c591a2-e77b-4550-9466-5dd2bf95c190
162025-08-28T22:03:02.484ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e1c591a2-e77b-4550-9466-5dd2bf95c190
172025-08-28T22:03:02.484ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-08-28T22:03:02.484ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
192025-08-28T22:03:02.484ZINFOcrucible-agent (worker): commit
202025-08-28T22:03:02.491ZINFOcrucible-agent (worker): ok!
212025-08-28T22:03:02.497ZINFOcrucible-agent (worker): SMF ok!
222025-08-28T22:03:02.599ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62780 req_id = b9c6ae38-d799-4619-871d-d2d6a209a806 response_code = 200 uri = /crucible/0/regions
232025-08-28T22:03:03.076ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62780 req_id = 549c0586-6511-4f11-bc3b-24b2039614d3 response_code = 200 uri = /crucible/0/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4
242025-08-28T22:03:03.080ZINFOcrucible-agent (datafile): path is "/data/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4"
252025-08-28T22:03:03.089ZINFOcrucible-agent (datafile): dataset is oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4
262025-08-28T22:03:03.097ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4@c31b42a6-2942-4d80-aa41-3218c533595f
272025-08-28T22:03:03.097ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4", "c31b42a6-2942-4d80-aa41-3218c533595f"]
282025-08-28T22:03:03.107ZINFOcrucible-agent (dropshot): request completed latency_us = 27735 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62780 req_id = 9bafd6e6-832d-4c7e-b44a-fd9ba41937de response_code = 200 uri = /crucible/0/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4/snapshots/c31b42a6-2942-4d80-aa41-3218c533595f
292025-08-28T22:03:03.111ZINFOcrucible-agent (datafile): path is "/data/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4"
302025-08-28T22:03:03.118ZINFOcrucible-agent (datafile): dataset is oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4
312025-08-28T22:03:03.126ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4@c31b42a6-2942-4d80-aa41-3218c533595f
322025-08-28T22:03:03.126ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4", "c31b42a6-2942-4d80-aa41-3218c533595f"]
332025-08-28T22:03:03.135ZINFOcrucible-agent (datafile): requesting running snapshot eb4f1458-9cfc-48a2-bf54-d4c984807fa4-c31b42a6-2942-4d80-aa41-3218c533595f state: Requested
342025-08-28T22:03:03.136ZINFOcrucible-agent (worker): applying SMF actions for region eb4f1458-9cfc-48a2-bf54-d4c984807fa4 running snapshot c31b42a6-2942-4d80-aa41-3218c533595f (state Requested)...
352025-08-28T22:03:03.136ZINFOcrucible-agent (dropshot): request completed latency_us = 24817 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62780 req_id = ed7551de-a422-4663-87f1-76ab4944e165 response_code = 200 uri = /crucible/0/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4/snapshots/c31b42a6-2942-4d80-aa41-3218c533595f/run
362025-08-28T22:03:03.138ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-eb4f1458-9cfc-48a2-bf54-d4c984807fa4-c31b42a6-2942-4d80-aa41-3218c533595f
372025-08-28T22:03:03.141ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-eb4f1458-9cfc-48a2-bf54-d4c984807fa4-c31b42a6-2942-4d80-aa41-3218c533595f
382025-08-28T22:03:03.152ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-eb4f1458-9cfc-48a2-bf54-d4c984807fa4-c31b42a6-2942-4d80-aa41-3218c533595f
392025-08-28T22:03:03.153ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4/.zfs/snapshot/c31b42a6-2942-4d80-aa41-3218c533595f
402025-08-28T22:03:03.153ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-08-28T22:03:03.153ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-08-28T22:03:03.153ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432025-08-28T22:03:03.153ZINFOcrucible-agent (worker): commit
442025-08-28T22:03:03.157ZINFOcrucible-agent (worker): ok!
452025-08-28T22:03:03.161ZINFOcrucible-agent (worker): SMF ok!
462025-08-28T22:03:03.161ZINFOcrucible-agent (datafile): region eb4f1458-9cfc-48a2-bf54-d4c984807fa4 running snapshot c31b42a6-2942-4d80-aa41-3218c533595f state: Requested -> Created
472025-08-28T22:03:03.344ZINFOcrucible-agent (datafile): path is "/data/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4"
482025-08-28T22:03:03.354ZINFOcrucible-agent (datafile): dataset is oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4
492025-08-28T22:03:03.363ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4@c31b42a6-2942-4d80-aa41-3218c533595f
502025-08-28T22:03:03.363ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1c2add25-73a7-4293-b69b-c31c9e5ca4b5/crucible/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4", "c31b42a6-2942-4d80-aa41-3218c533595f"]
512025-08-28T22:03:03.371ZINFOcrucible-agent (dropshot): request completed latency_us = 26642 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62780 req_id = 0c1ca4d3-1f32-4cc5-922e-91926ae14130 response_code = 200 uri = /crucible/0/regions/eb4f1458-9cfc-48a2-bf54-d4c984807fa4/snapshots/c31b42a6-2942-4d80-aa41-3218c533595f/run