Max level shown:
12025-09-24T17:03:05.664ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:48850
22025-09-24T17:03:05.665ZINFOcrucible-agent (datafile): region 6b0e373e-562f-49f9-96aa-f3620069a768 state: Requested
32025-09-24T17:03:05.665ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-09-24T17:03:05.666ZINFOcrucible-agent (dropshot): request completed latency_us = 992 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48850 req_id = 30dfbcac-a7e7-42ed-a139-564a9adf8b20 response_code = 200 uri = /crucible/0/regions
52025-09-24T17:03:05.678ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/6b0e373e-562f-49f9-96aa-f3620069a768
62025-09-24T17:03:05.678ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/6b0e373e-562f-49f9-96aa-f3620069a768
72025-09-24T17:03:05.727ZINFOcrucible-agent (worker): creating region Region { id: RegionId("6b0e373e-562f-49f9-96aa-f3620069a768"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/6b0e373e-562f-49f9-96aa-f3620069a768" region = 6b0e373e-562f-49f9-96aa-f3620069a768
82025-09-24T17:03:05.729ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "6b0e373e-562f-49f9-96aa-f3620069a768" "--data" "/data/regions/6b0e373e-562f-49f9-96aa-f3620069a768" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 6b0e373e-562f-49f9-96aa-f3620069a768
92025-09-24T17:03:05.852ZINFOcrucible-agent (worker): region files created ok region = 6b0e373e-562f-49f9-96aa-f3620069a768
102025-09-24T17:03:05.852ZINFOcrucible-agent (datafile): region 6b0e373e-562f-49f9-96aa-f3620069a768 state: Requested -> Created
112025-09-24T17:03:05.852ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-09-24T17:03:05.853ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-6b0e373e-562f-49f9-96aa-f3620069a768
132025-09-24T17:03:05.870ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-6b0e373e-562f-49f9-96aa-f3620069a768
142025-09-24T17:03:05.890ZINFOcrucible-agent (worker): creating config property group
152025-09-24T17:03:05.892ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-6b0e373e-562f-49f9-96aa-f3620069a768
162025-09-24T17:03:05.893ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/6b0e373e-562f-49f9-96aa-f3620069a768
172025-09-24T17:03:05.893ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-09-24T17:03:05.893ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192025-09-24T17:03:05.893ZINFOcrucible-agent (worker): commit
202025-09-24T17:03:05.896ZINFOcrucible-agent (worker): ok!
212025-09-24T17:03:05.902ZINFOcrucible-agent (worker): SMF ok!
222025-09-24T17:03:06.012ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48850 req_id = c6bd4be8-50ab-49f4-aac2-0ef106889116 response_code = 200 uri = /crucible/0/regions
232025-09-24T17:03:07.998ZINFOcrucible-agent (dropshot): request completed latency_us = 46 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48850 req_id = 17a66338-8784-4248-939d-302dd0688228 response_code = 200 uri = /crucible/0/regions/7a289382-695d-48ca-8dc7-96c191fee5b4
242025-09-24T17:03:08.001ZINFOcrucible-agent (datafile): path is "/data/regions/7a289382-695d-48ca-8dc7-96c191fee5b4"
252025-09-24T17:03:08.011ZINFOcrucible-agent (datafile): dataset is oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/7a289382-695d-48ca-8dc7-96c191fee5b4
262025-09-24T17:03:08.019ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/7a289382-695d-48ca-8dc7-96c191fee5b4@a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3
272025-09-24T17:03:08.019ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/7a289382-695d-48ca-8dc7-96c191fee5b4", "a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3"]
282025-09-24T17:03:08.031ZINFOcrucible-agent (dropshot): request completed latency_us = 29521 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48850 req_id = d95c43d6-b52c-43ef-873a-74d0f64c9c8b response_code = 200 uri = /crucible/0/regions/7a289382-695d-48ca-8dc7-96c191fee5b4/snapshots/a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3
292025-09-24T17:03:08.034ZINFOcrucible-agent (datafile): path is "/data/regions/7a289382-695d-48ca-8dc7-96c191fee5b4"
302025-09-24T17:03:08.043ZINFOcrucible-agent (datafile): dataset is oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/7a289382-695d-48ca-8dc7-96c191fee5b4
312025-09-24T17:03:08.051ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/7a289382-695d-48ca-8dc7-96c191fee5b4@a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3
322025-09-24T17:03:08.051ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/7a289382-695d-48ca-8dc7-96c191fee5b4", "a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3"]
332025-09-24T17:03:08.061ZINFOcrucible-agent (datafile): requesting running snapshot 7a289382-695d-48ca-8dc7-96c191fee5b4-a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3 state: Requested
342025-09-24T17:03:08.061ZINFOcrucible-agent (worker): applying SMF actions for region 7a289382-695d-48ca-8dc7-96c191fee5b4 running snapshot a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3 (state Requested)...
352025-09-24T17:03:08.061ZINFOcrucible-agent (dropshot): request completed latency_us = 27249 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48850 req_id = aa817c37-7cd0-42ae-b561-ad00481d10e1 response_code = 200 uri = /crucible/0/regions/7a289382-695d-48ca-8dc7-96c191fee5b4/snapshots/a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3/run
362025-09-24T17:03:08.064ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-7a289382-695d-48ca-8dc7-96c191fee5b4-a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3
372025-09-24T17:03:08.066ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-7a289382-695d-48ca-8dc7-96c191fee5b4-a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3
382025-09-24T17:03:08.077ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-7a289382-695d-48ca-8dc7-96c191fee5b4-a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3
392025-09-24T17:03:08.077ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/7a289382-695d-48ca-8dc7-96c191fee5b4/.zfs/snapshot/a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3
402025-09-24T17:03:08.077ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-09-24T17:03:08.077ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-09-24T17:03:08.077ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432025-09-24T17:03:08.077ZINFOcrucible-agent (worker): commit
442025-09-24T17:03:08.081ZINFOcrucible-agent (worker): ok!
452025-09-24T17:03:08.085ZINFOcrucible-agent (worker): SMF ok!
462025-09-24T17:03:08.085ZINFOcrucible-agent (datafile): region 7a289382-695d-48ca-8dc7-96c191fee5b4 running snapshot a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3 state: Requested -> Created
472025-09-24T17:03:08.275ZINFOcrucible-agent (datafile): path is "/data/regions/7a289382-695d-48ca-8dc7-96c191fee5b4"
482025-09-24T17:03:08.283ZINFOcrucible-agent (datafile): dataset is oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/7a289382-695d-48ca-8dc7-96c191fee5b4
492025-09-24T17:03:08.292ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/7a289382-695d-48ca-8dc7-96c191fee5b4@a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3
502025-09-24T17:03:08.292ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_05ccfef6-651c-4107-ade5-369be6ca9a62/crucible/regions/7a289382-695d-48ca-8dc7-96c191fee5b4", "a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3"]
512025-09-24T17:03:08.299ZINFOcrucible-agent (dropshot): request completed latency_us = 24582 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48850 req_id = 349fcb54-2855-4c51-b554-e6d5d85eb7c5 response_code = 200 uri = /crucible/0/regions/7a289382-695d-48ca-8dc7-96c191fee5b4/snapshots/a7312d9d-4bb3-44a5-9ec5-eacb1c4aefd3/run