Max level shown:
12025-01-29T05:19:27.648ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::c]:59036
22025-01-29T05:19:27.648ZINFOcrucible-agent (datafile): region 4e2add20-2751-4519-bfaa-c2076671c96d state: Requested
32025-01-29T05:19:27.649ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-01-29T05:19:27.649ZINFOcrucible-agent (dropshot): request completed latency_us = 760 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:59036 req_id = cc8959b3-6c5b-4081-9eeb-040f22fc1137 response_code = 200 uri = /crucible/0/regions
52025-01-29T05:19:27.659ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/4e2add20-2751-4519-bfaa-c2076671c96d
62025-01-29T05:19:27.659ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/4e2add20-2751-4519-bfaa-c2076671c96d
72025-01-29T05:19:27.726ZINFOcrucible-agent (worker): creating region Region { id: RegionId("4e2add20-2751-4519-bfaa-c2076671c96d"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/4e2add20-2751-4519-bfaa-c2076671c96d" region = 4e2add20-2751-4519-bfaa-c2076671c96d
82025-01-29T05:19:27.728ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "4e2add20-2751-4519-bfaa-c2076671c96d" "--data" "/data/regions/4e2add20-2751-4519-bfaa-c2076671c96d" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 4e2add20-2751-4519-bfaa-c2076671c96d
92025-01-29T05:19:27.831ZINFOcrucible-agent (worker): region files created ok region = 4e2add20-2751-4519-bfaa-c2076671c96d
102025-01-29T05:19:27.831ZINFOcrucible-agent (datafile): region 4e2add20-2751-4519-bfaa-c2076671c96d state: Requested -> Created
112025-01-29T05:19:27.831ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-01-29T05:19:27.832ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-4e2add20-2751-4519-bfaa-c2076671c96d
132025-01-29T05:19:27.840ZINFOcrucible-agent (dropshot): request completed latency_us = 111 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:59036 req_id = fa7912a8-ecc3-4351-90b0-0f6d3381fe78 response_code = 200 uri = /crucible/0/regions
142025-01-29T05:19:27.848ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-4e2add20-2751-4519-bfaa-c2076671c96d
152025-01-29T05:19:27.877ZINFOcrucible-agent (worker): creating config property group
162025-01-29T05:19:27.883ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4e2add20-2751-4519-bfaa-c2076671c96d
172025-01-29T05:19:27.883ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4e2add20-2751-4519-bfaa-c2076671c96d
182025-01-29T05:19:27.883ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-01-29T05:19:27.883ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
202025-01-29T05:19:27.883ZINFOcrucible-agent (worker): commit
212025-01-29T05:19:27.890ZINFOcrucible-agent (worker): ok!
222025-01-29T05:19:27.901ZINFOcrucible-agent (worker): SMF ok!
232025-01-29T05:19:29.416ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:59036 req_id = a2f9edc4-649d-4300-9472-24396c2f195d response_code = 200 uri = /crucible/0/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0
242025-01-29T05:19:29.420ZINFOcrucible-agent (datafile): path is "/data/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0"
252025-01-29T05:19:29.428ZINFOcrucible-agent (datafile): dataset is oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0
262025-01-29T05:19:29.436ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0@963521c2-e0e4-4075-b9d1-fb8918982b02
272025-01-29T05:19:29.437ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0", "963521c2-e0e4-4075-b9d1-fb8918982b02"]
282025-01-29T05:19:29.451ZINFOcrucible-agent (dropshot): request completed latency_us = 31089 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:59036 req_id = d906582a-a81a-4bed-852f-05b255e3c12c response_code = 200 uri = /crucible/0/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0/snapshots/963521c2-e0e4-4075-b9d1-fb8918982b02
292025-01-29T05:19:29.456ZINFOcrucible-agent (datafile): path is "/data/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0"
302025-01-29T05:19:29.465ZINFOcrucible-agent (datafile): dataset is oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0
312025-01-29T05:19:29.474ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0@963521c2-e0e4-4075-b9d1-fb8918982b02
322025-01-29T05:19:29.474ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0", "963521c2-e0e4-4075-b9d1-fb8918982b02"]
332025-01-29T05:19:29.482ZINFOcrucible-agent (datafile): requesting running snapshot 80fbc4ef-94f9-458e-afe2-71e0c7cc16e0-963521c2-e0e4-4075-b9d1-fb8918982b02 state: Requested
342025-01-29T05:19:29.483ZINFOcrucible-agent (worker): applying SMF actions for region 80fbc4ef-94f9-458e-afe2-71e0c7cc16e0 running snapshot 963521c2-e0e4-4075-b9d1-fb8918982b02 (state Requested)...
352025-01-29T05:19:29.483ZINFOcrucible-agent (dropshot): request completed latency_us = 26874 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:59036 req_id = 44bddfd3-ced9-40a2-98d9-c4be2ab0c10e response_code = 200 uri = /crucible/0/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0/snapshots/963521c2-e0e4-4075-b9d1-fb8918982b02/run
362025-01-29T05:19:29.486ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-80fbc4ef-94f9-458e-afe2-71e0c7cc16e0-963521c2-e0e4-4075-b9d1-fb8918982b02
372025-01-29T05:19:29.490ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-80fbc4ef-94f9-458e-afe2-71e0c7cc16e0-963521c2-e0e4-4075-b9d1-fb8918982b02
382025-01-29T05:19:29.508ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-80fbc4ef-94f9-458e-afe2-71e0c7cc16e0-963521c2-e0e4-4075-b9d1-fb8918982b02
392025-01-29T05:19:29.508ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0/.zfs/snapshot/963521c2-e0e4-4075-b9d1-fb8918982b02
402025-01-29T05:19:29.508ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-01-29T05:19:29.508ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-01-29T05:19:29.509ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432025-01-29T05:19:29.509ZINFOcrucible-agent (worker): commit
442025-01-29T05:19:29.515ZINFOcrucible-agent (worker): ok!
452025-01-29T05:19:29.520ZINFOcrucible-agent (worker): SMF ok!
462025-01-29T05:19:29.520ZINFOcrucible-agent (datafile): region 80fbc4ef-94f9-458e-afe2-71e0c7cc16e0 running snapshot 963521c2-e0e4-4075-b9d1-fb8918982b02 state: Requested -> Created
472025-01-29T05:19:29.733ZINFOcrucible-agent (datafile): path is "/data/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0"
482025-01-29T05:19:29.742ZINFOcrucible-agent (datafile): dataset is oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0
492025-01-29T05:19:29.750ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0@963521c2-e0e4-4075-b9d1-fb8918982b02
502025-01-29T05:19:29.750ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_001e8549-f6b0-425b-884f-7cb9db67c1af/crucible/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0", "963521c2-e0e4-4075-b9d1-fb8918982b02"]
512025-01-29T05:19:29.760ZINFOcrucible-agent (dropshot): request completed latency_us = 27225 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:59036 req_id = a7457636-f6ae-43bb-9372-f4847ae4d2dc response_code = 200 uri = /crucible/0/regions/80fbc4ef-94f9-458e-afe2-71e0c7cc16e0/snapshots/963521c2-e0e4-4075-b9d1-fb8918982b02/run