Max level shown:
12025-06-04T04:47:56.691ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:58220
22025-06-04T04:47:56.691ZINFOcrucible-agent (datafile): region 2386a9f6-4454-40b1-b850-c117bc294f2d state: Requested
32025-06-04T04:47:56.692ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-06-04T04:47:56.692ZINFOcrucible-agent (dropshot): request completed latency_us = 683 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58220 req_id = 93f564ef-59a0-42af-9666-31b8fe4589fd response_code = 200 uri = /crucible/0/regions
52025-06-04T04:47:56.704ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/2386a9f6-4454-40b1-b850-c117bc294f2d
62025-06-04T04:47:56.704ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/2386a9f6-4454-40b1-b850-c117bc294f2d
72025-06-04T04:47:56.754ZINFOcrucible-agent (worker): creating region Region { id: RegionId("2386a9f6-4454-40b1-b850-c117bc294f2d"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/2386a9f6-4454-40b1-b850-c117bc294f2d" region = 2386a9f6-4454-40b1-b850-c117bc294f2d
82025-06-04T04:47:56.756ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "2386a9f6-4454-40b1-b850-c117bc294f2d" "--data" "/data/regions/2386a9f6-4454-40b1-b850-c117bc294f2d" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 2386a9f6-4454-40b1-b850-c117bc294f2d
92025-06-04T04:47:56.872ZINFOcrucible-agent (worker): region files created ok region = 2386a9f6-4454-40b1-b850-c117bc294f2d
102025-06-04T04:47:56.872ZINFOcrucible-agent (datafile): region 2386a9f6-4454-40b1-b850-c117bc294f2d state: Requested -> Created
112025-06-04T04:47:56.872ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-06-04T04:47:56.873ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-2386a9f6-4454-40b1-b850-c117bc294f2d
132025-06-04T04:47:56.881ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58220 req_id = ad9132b0-2b53-4d39-af43-b65b6dbfc4cb response_code = 200 uri = /crucible/0/regions
142025-06-04T04:47:56.890ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-2386a9f6-4454-40b1-b850-c117bc294f2d
152025-06-04T04:47:56.912ZINFOcrucible-agent (worker): creating config property group
162025-06-04T04:47:56.915ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-2386a9f6-4454-40b1-b850-c117bc294f2d
172025-06-04T04:47:56.915ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2386a9f6-4454-40b1-b850-c117bc294f2d
182025-06-04T04:47:56.915ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-06-04T04:47:56.915ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
202025-06-04T04:47:56.916ZINFOcrucible-agent (worker): commit
212025-06-04T04:47:56.919ZINFOcrucible-agent (worker): ok!
222025-06-04T04:47:56.924ZINFOcrucible-agent (worker): SMF ok!
232025-06-04T04:47:58.609ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:58220 req_id = d1456a59-86ca-4dc2-a5f5-44f22db3079d response_code = 200 uri = /crucible/0/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e
242025-06-04T04:47:58.613ZINFOcrucible-agent (datafile): path is "/data/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e"
252025-06-04T04:47:58.623ZINFOcrucible-agent (datafile): dataset is oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e
262025-06-04T04:47:58.632ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e@eb58ef6f-7350-42ab-884b-52e09e64904c
272025-06-04T04:47:58.632ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e", "eb58ef6f-7350-42ab-884b-52e09e64904c"]
282025-06-04T04:47:58.644ZINFOcrucible-agent (dropshot): request completed latency_us = 30879 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:58220 req_id = ed0bfabb-3669-4d1f-b087-9f743872004f response_code = 200 uri = /crucible/0/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e/snapshots/eb58ef6f-7350-42ab-884b-52e09e64904c
292025-06-04T04:47:58.647ZINFOcrucible-agent (datafile): path is "/data/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e"
302025-06-04T04:47:58.657ZINFOcrucible-agent (datafile): dataset is oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e
312025-06-04T04:47:58.666ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e@eb58ef6f-7350-42ab-884b-52e09e64904c
322025-06-04T04:47:58.666ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e", "eb58ef6f-7350-42ab-884b-52e09e64904c"]
332025-06-04T04:47:58.677ZINFOcrucible-agent (datafile): requesting running snapshot f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e-eb58ef6f-7350-42ab-884b-52e09e64904c state: Requested
342025-06-04T04:47:58.677ZINFOcrucible-agent (worker): applying SMF actions for region f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e running snapshot eb58ef6f-7350-42ab-884b-52e09e64904c (state Requested)...
352025-06-04T04:47:58.677ZINFOcrucible-agent (dropshot): request completed latency_us = 29729 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58220 req_id = 4e95d33f-54af-4f4c-b64c-68c37cacdc16 response_code = 200 uri = /crucible/0/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e/snapshots/eb58ef6f-7350-42ab-884b-52e09e64904c/run
362025-06-04T04:47:58.680ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e-eb58ef6f-7350-42ab-884b-52e09e64904c
372025-06-04T04:47:58.683ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e-eb58ef6f-7350-42ab-884b-52e09e64904c
382025-06-04T04:47:58.695ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e-eb58ef6f-7350-42ab-884b-52e09e64904c
392025-06-04T04:47:58.695ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e/.zfs/snapshot/eb58ef6f-7350-42ab-884b-52e09e64904c
402025-06-04T04:47:58.695ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-06-04T04:47:58.695ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-06-04T04:47:58.695ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
432025-06-04T04:47:58.695ZINFOcrucible-agent (worker): commit
442025-06-04T04:47:58.700ZINFOcrucible-agent (worker): ok!
452025-06-04T04:47:58.705ZINFOcrucible-agent (worker): SMF ok!
462025-06-04T04:47:58.705ZINFOcrucible-agent (datafile): region f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e running snapshot eb58ef6f-7350-42ab-884b-52e09e64904c state: Requested -> Created
472025-06-04T04:47:58.909ZINFOcrucible-agent (datafile): path is "/data/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e"
482025-06-04T04:47:58.918ZINFOcrucible-agent (datafile): dataset is oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e
492025-06-04T04:47:58.927ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e@eb58ef6f-7350-42ab-884b-52e09e64904c
502025-06-04T04:47:58.927ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e93f79a6-c560-4e72-91ff-411bc721723c/crucible/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e", "eb58ef6f-7350-42ab-884b-52e09e64904c"]
512025-06-04T04:47:58.936ZINFOcrucible-agent (dropshot): request completed latency_us = 26942 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58220 req_id = 78893d77-e52a-4869-ab06-adeee5fde7b6 response_code = 200 uri = /crucible/0/regions/f3e2f435-ce1f-4c1e-8ce2-0197d0d6d92e/snapshots/eb58ef6f-7350-42ab-884b-52e09e64904c/run