Max level shown:
12025-08-12T22:54:40.627ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::a]:42866
22025-08-12T22:54:40.627ZINFOcrucible-agent (datafile): region 2f1fa16a-b42a-405b-bbe1-837346c55ce6 state: Requested
32025-08-12T22:54:40.627ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-08-12T22:54:40.627ZINFOcrucible-agent (dropshot): request completed latency_us = 857 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42866 req_id = 016928fd-a4aa-4673-957f-1b5d10418652 response_code = 200 uri = /crucible/0/regions
52025-08-12T22:54:40.639ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/2f1fa16a-b42a-405b-bbe1-837346c55ce6
62025-08-12T22:54:40.639ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/2f1fa16a-b42a-405b-bbe1-837346c55ce6
72025-08-12T22:54:40.702ZINFOcrucible-agent (worker): creating region Region { id: RegionId("2f1fa16a-b42a-405b-bbe1-837346c55ce6"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/2f1fa16a-b42a-405b-bbe1-837346c55ce6" region = 2f1fa16a-b42a-405b-bbe1-837346c55ce6
82025-08-12T22:54:40.702ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "2f1fa16a-b42a-405b-bbe1-837346c55ce6" "--data" "/data/regions/2f1fa16a-b42a-405b-bbe1-837346c55ce6" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 2f1fa16a-b42a-405b-bbe1-837346c55ce6
92025-08-12T22:54:40.851ZINFOcrucible-agent (worker): region files created ok region = 2f1fa16a-b42a-405b-bbe1-837346c55ce6
102025-08-12T22:54:40.851ZINFOcrucible-agent (datafile): region 2f1fa16a-b42a-405b-bbe1-837346c55ce6 state: Requested -> Created
112025-08-12T22:54:40.851ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-08-12T22:54:40.853ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-2f1fa16a-b42a-405b-bbe1-837346c55ce6
132025-08-12T22:54:40.861ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42866 req_id = b79e5dc9-5dfc-42ab-b940-67ba7f5298cb response_code = 200 uri = /crucible/0/regions
142025-08-12T22:54:40.877ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-2f1fa16a-b42a-405b-bbe1-837346c55ce6
152025-08-12T22:54:40.903ZINFOcrucible-agent (worker): creating config property group
162025-08-12T22:54:40.910ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-2f1fa16a-b42a-405b-bbe1-837346c55ce6
172025-08-12T22:54:40.910ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2f1fa16a-b42a-405b-bbe1-837346c55ce6
182025-08-12T22:54:40.911ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-08-12T22:54:40.911ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
202025-08-12T22:54:40.911ZINFOcrucible-agent (worker): commit
212025-08-12T22:54:40.916ZINFOcrucible-agent (worker): ok!
222025-08-12T22:54:40.926ZINFOcrucible-agent (worker): SMF ok!
232025-08-12T22:54:43.430ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42866 req_id = 6bf91d5f-6446-4c9e-8987-2a74351e5108 response_code = 200 uri = /crucible/0/regions/bac812b4-2853-4eca-b0a5-c2766304b43c
242025-08-12T22:54:43.434ZINFOcrucible-agent (datafile): path is "/data/regions/bac812b4-2853-4eca-b0a5-c2766304b43c"
252025-08-12T22:54:43.443ZINFOcrucible-agent (datafile): dataset is oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/bac812b4-2853-4eca-b0a5-c2766304b43c
262025-08-12T22:54:43.453ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/bac812b4-2853-4eca-b0a5-c2766304b43c@ae9da4d5-5156-49bb-9c03-074bcb3c906d
272025-08-12T22:54:43.453ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/bac812b4-2853-4eca-b0a5-c2766304b43c", "ae9da4d5-5156-49bb-9c03-074bcb3c906d"]
282025-08-12T22:54:43.464ZINFOcrucible-agent (dropshot): request completed latency_us = 30405 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42866 req_id = 59b8d1e4-bf95-41b1-a7fd-e0de806fa8a8 response_code = 200 uri = /crucible/0/regions/bac812b4-2853-4eca-b0a5-c2766304b43c/snapshots/ae9da4d5-5156-49bb-9c03-074bcb3c906d
292025-08-12T22:54:43.468ZINFOcrucible-agent (datafile): path is "/data/regions/bac812b4-2853-4eca-b0a5-c2766304b43c"
302025-08-12T22:54:43.476ZINFOcrucible-agent (datafile): dataset is oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/bac812b4-2853-4eca-b0a5-c2766304b43c
312025-08-12T22:54:43.485ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/bac812b4-2853-4eca-b0a5-c2766304b43c@ae9da4d5-5156-49bb-9c03-074bcb3c906d
322025-08-12T22:54:43.485ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/bac812b4-2853-4eca-b0a5-c2766304b43c", "ae9da4d5-5156-49bb-9c03-074bcb3c906d"]
332025-08-12T22:54:43.495ZINFOcrucible-agent (datafile): requesting running snapshot bac812b4-2853-4eca-b0a5-c2766304b43c-ae9da4d5-5156-49bb-9c03-074bcb3c906d state: Requested
342025-08-12T22:54:43.495ZINFOcrucible-agent (worker): applying SMF actions for region bac812b4-2853-4eca-b0a5-c2766304b43c running snapshot ae9da4d5-5156-49bb-9c03-074bcb3c906d (state Requested)...
352025-08-12T22:54:43.495ZINFOcrucible-agent (dropshot): request completed latency_us = 27542 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42866 req_id = 9d1e8a8f-2342-4c46-97e4-ce78210bf50c response_code = 200 uri = /crucible/0/regions/bac812b4-2853-4eca-b0a5-c2766304b43c/snapshots/ae9da4d5-5156-49bb-9c03-074bcb3c906d/run
362025-08-12T22:54:43.498ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-bac812b4-2853-4eca-b0a5-c2766304b43c-ae9da4d5-5156-49bb-9c03-074bcb3c906d
372025-08-12T22:54:43.504ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-bac812b4-2853-4eca-b0a5-c2766304b43c-ae9da4d5-5156-49bb-9c03-074bcb3c906d
382025-08-12T22:54:43.522ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-bac812b4-2853-4eca-b0a5-c2766304b43c-ae9da4d5-5156-49bb-9c03-074bcb3c906d
392025-08-12T22:54:43.522ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/bac812b4-2853-4eca-b0a5-c2766304b43c/.zfs/snapshot/ae9da4d5-5156-49bb-9c03-074bcb3c906d
402025-08-12T22:54:43.523ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-08-12T22:54:43.523ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-08-12T22:54:43.523ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
432025-08-12T22:54:43.523ZINFOcrucible-agent (worker): commit
442025-08-12T22:54:43.527ZINFOcrucible-agent (worker): ok!
452025-08-12T22:54:43.535ZINFOcrucible-agent (worker): SMF ok!
462025-08-12T22:54:43.535ZINFOcrucible-agent (datafile): region bac812b4-2853-4eca-b0a5-c2766304b43c running snapshot ae9da4d5-5156-49bb-9c03-074bcb3c906d state: Requested -> Created
472025-08-12T22:54:43.756ZINFOcrucible-agent (datafile): path is "/data/regions/bac812b4-2853-4eca-b0a5-c2766304b43c"
482025-08-12T22:54:43.765ZINFOcrucible-agent (datafile): dataset is oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/bac812b4-2853-4eca-b0a5-c2766304b43c
492025-08-12T22:54:43.773ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/bac812b4-2853-4eca-b0a5-c2766304b43c@ae9da4d5-5156-49bb-9c03-074bcb3c906d
502025-08-12T22:54:43.774ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d3215026-98bd-479c-99b1-6e348ebc97e9/crucible/regions/bac812b4-2853-4eca-b0a5-c2766304b43c", "ae9da4d5-5156-49bb-9c03-074bcb3c906d"]
512025-08-12T22:54:43.782ZINFOcrucible-agent (dropshot): request completed latency_us = 25972 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42866 req_id = a295f832-0656-4e17-aa46-928eab3ddeb4 response_code = 200 uri = /crucible/0/regions/bac812b4-2853-4eca-b0a5-c2766304b43c/snapshots/ae9da4d5-5156-49bb-9c03-074bcb3c906d/run