Max level shown:
12025-12-17T00:21:58.776ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::a]:62630
22025-12-17T00:21:58.776ZINFOcrucible-agent (datafile): region 39927d59-ec0f-4ce5-9f29-47579bafcaa1 state: Requested
32025-12-17T00:21:58.777ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-12-17T00:21:58.777ZINFOcrucible-agent (dropshot): request completed latency_us = 604 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62630 req_id = 4fe83c53-2f6e-4cf5-a823-05c5842847e5 response_code = 200 uri = /crucible/0/regions
52025-12-17T00:21:58.786ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/39927d59-ec0f-4ce5-9f29-47579bafcaa1
62025-12-17T00:21:58.786ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/39927d59-ec0f-4ce5-9f29-47579bafcaa1
72025-12-17T00:21:58.837ZINFOcrucible-agent (worker): creating region Region { id: RegionId("39927d59-ec0f-4ce5-9f29-47579bafcaa1"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/39927d59-ec0f-4ce5-9f29-47579bafcaa1" region = 39927d59-ec0f-4ce5-9f29-47579bafcaa1
82025-12-17T00:21:58.837ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "39927d59-ec0f-4ce5-9f29-47579bafcaa1" "--data" "/data/regions/39927d59-ec0f-4ce5-9f29-47579bafcaa1" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 39927d59-ec0f-4ce5-9f29-47579bafcaa1
92025-12-17T00:21:58.940ZINFOcrucible-agent (worker): region files created ok region = 39927d59-ec0f-4ce5-9f29-47579bafcaa1
102025-12-17T00:21:58.940ZINFOcrucible-agent (datafile): region 39927d59-ec0f-4ce5-9f29-47579bafcaa1 state: Requested -> Created
112025-12-17T00:21:58.940ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-12-17T00:21:58.959ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-39927d59-ec0f-4ce5-9f29-47579bafcaa1
132025-12-17T00:21:58.965ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-39927d59-ec0f-4ce5-9f29-47579bafcaa1
142025-12-17T00:21:58.974ZINFOcrucible-agent (worker): creating config property group
152025-12-17T00:21:58.977ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-39927d59-ec0f-4ce5-9f29-47579bafcaa1
162025-12-17T00:21:58.978ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/39927d59-ec0f-4ce5-9f29-47579bafcaa1
172025-12-17T00:21:58.978ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-12-17T00:21:58.978ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
192025-12-17T00:21:58.978ZINFOcrucible-agent (worker): commit
202025-12-17T00:21:58.980ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62630 req_id = 3a9616ff-5c5f-4787-a769-7ca4fabf5042 response_code = 200 uri = /crucible/0/regions
212025-12-17T00:21:58.982ZINFOcrucible-agent (worker): ok!
222025-12-17T00:21:58.987ZINFOcrucible-agent (worker): SMF ok!
232025-12-17T00:21:59.839ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62630 req_id = e5ea93a4-6e01-40c9-aef2-118e5ee5177b response_code = 200 uri = /crucible/0/regions/35a535f2-f941-4895-bb5f-375d7d33deb1
242025-12-17T00:21:59.844ZINFOcrucible-agent (datafile): path is "/data/regions/35a535f2-f941-4895-bb5f-375d7d33deb1"
252025-12-17T00:21:59.851ZINFOcrucible-agent (datafile): dataset is oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/35a535f2-f941-4895-bb5f-375d7d33deb1
262025-12-17T00:21:59.857ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/35a535f2-f941-4895-bb5f-375d7d33deb1@b06b72a6-bbf9-49d7-9253-c4ffa8bbba75
272025-12-17T00:21:59.857ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/35a535f2-f941-4895-bb5f-375d7d33deb1", "b06b72a6-bbf9-49d7-9253-c4ffa8bbba75"]
282025-12-17T00:21:59.865ZINFOcrucible-agent (dropshot): request completed latency_us = 20543 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:62630 req_id = b6aeccf4-bdc1-44d1-8fcc-57948f86bd5f response_code = 200 uri = /crucible/0/regions/35a535f2-f941-4895-bb5f-375d7d33deb1/snapshots/b06b72a6-bbf9-49d7-9253-c4ffa8bbba75
292025-12-17T00:21:59.870ZINFOcrucible-agent (datafile): path is "/data/regions/35a535f2-f941-4895-bb5f-375d7d33deb1"
302025-12-17T00:21:59.875ZINFOcrucible-agent (datafile): dataset is oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/35a535f2-f941-4895-bb5f-375d7d33deb1
312025-12-17T00:21:59.882ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/35a535f2-f941-4895-bb5f-375d7d33deb1@b06b72a6-bbf9-49d7-9253-c4ffa8bbba75
322025-12-17T00:21:59.882ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/35a535f2-f941-4895-bb5f-375d7d33deb1", "b06b72a6-bbf9-49d7-9253-c4ffa8bbba75"]
332025-12-17T00:21:59.889ZINFOcrucible-agent (datafile): requesting running snapshot 35a535f2-f941-4895-bb5f-375d7d33deb1-b06b72a6-bbf9-49d7-9253-c4ffa8bbba75 state: Requested
342025-12-17T00:21:59.889ZINFOcrucible-agent (dropshot): request completed latency_us = 18904 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62630 req_id = 57be7aad-91f2-4864-a45c-a55c415cf0cc response_code = 200 uri = /crucible/0/regions/35a535f2-f941-4895-bb5f-375d7d33deb1/snapshots/b06b72a6-bbf9-49d7-9253-c4ffa8bbba75/run
352025-12-17T00:21:59.889ZINFOcrucible-agent (worker): applying SMF actions for region 35a535f2-f941-4895-bb5f-375d7d33deb1 running snapshot b06b72a6-bbf9-49d7-9253-c4ffa8bbba75 (state Requested)...
362025-12-17T00:21:59.893ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-35a535f2-f941-4895-bb5f-375d7d33deb1-b06b72a6-bbf9-49d7-9253-c4ffa8bbba75
372025-12-17T00:21:59.902ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-35a535f2-f941-4895-bb5f-375d7d33deb1-b06b72a6-bbf9-49d7-9253-c4ffa8bbba75
382025-12-17T00:21:59.910ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-35a535f2-f941-4895-bb5f-375d7d33deb1-b06b72a6-bbf9-49d7-9253-c4ffa8bbba75
392025-12-17T00:21:59.911ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/35a535f2-f941-4895-bb5f-375d7d33deb1/.zfs/snapshot/b06b72a6-bbf9-49d7-9253-c4ffa8bbba75
402025-12-17T00:21:59.911ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-12-17T00:21:59.911ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-12-17T00:21:59.911ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
432025-12-17T00:21:59.911ZINFOcrucible-agent (worker): commit
442025-12-17T00:21:59.915ZINFOcrucible-agent (worker): ok!
452025-12-17T00:21:59.919ZINFOcrucible-agent (worker): SMF ok!
462025-12-17T00:21:59.919ZINFOcrucible-agent (datafile): region 35a535f2-f941-4895-bb5f-375d7d33deb1 running snapshot b06b72a6-bbf9-49d7-9253-c4ffa8bbba75 state: Requested -> Created
472025-12-17T00:22:00.239ZINFOcrucible-agent (datafile): path is "/data/regions/35a535f2-f941-4895-bb5f-375d7d33deb1"
482025-12-17T00:22:00.245ZINFOcrucible-agent (datafile): dataset is oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/35a535f2-f941-4895-bb5f-375d7d33deb1
492025-12-17T00:22:00.250ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/35a535f2-f941-4895-bb5f-375d7d33deb1@b06b72a6-bbf9-49d7-9253-c4ffa8bbba75
502025-12-17T00:22:00.250ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_94d1ff98-7cfa-4059-96ef-a12373b703ba/crucible/regions/35a535f2-f941-4895-bb5f-375d7d33deb1", "b06b72a6-bbf9-49d7-9253-c4ffa8bbba75"]
512025-12-17T00:22:00.255ZINFOcrucible-agent (dropshot): request completed latency_us = 15736 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:62630 req_id = 6f470f35-619e-4606-b99f-a63f23296dfb response_code = 200 uri = /crucible/0/regions/35a535f2-f941-4895-bb5f-375d7d33deb1/snapshots/b06b72a6-bbf9-49d7-9253-c4ffa8bbba75/run