Max level shown:
12026-08-29T21:49:46.322ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::b]:59418
22026-08-29T21:49:46.322ZINFOcrucible-agent (datafile): region 43cab87f-da7f-46b7-be54-ca199098e92c state: Requested
32026-08-29T21:49:46.323ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-08-29T21:49:46.323ZINFOcrucible-agent (dropshot): request completed latency_us = 678 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:59418 req_id = cf223495-fab9-49cc-aed3-fd4036264fe5 response_code = 200 uri = /crucible/0/regions
52026-08-29T21:49:46.332ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/43cab87f-da7f-46b7-be54-ca199098e92c
62026-08-29T21:49:46.332ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/43cab87f-da7f-46b7-be54-ca199098e92c
72026-08-29T21:49:46.387ZINFOcrucible-agent (worker): creating region Region { id: RegionId("43cab87f-da7f-46b7-be54-ca199098e92c"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/43cab87f-da7f-46b7-be54-ca199098e92c" region = 43cab87f-da7f-46b7-be54-ca199098e92c
82026-08-29T21:49:46.387ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "43cab87f-da7f-46b7-be54-ca199098e92c" "--data" "/data/regions/43cab87f-da7f-46b7-be54-ca199098e92c" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 43cab87f-da7f-46b7-be54-ca199098e92c
92026-08-29T21:49:46.464ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:59418 req_id = ec256359-42b8-41f2-8bca-f1204b1e3f85 response_code = 200 uri = /crucible/0/regions
102026-08-29T21:49:46.465ZINFOcrucible-agent (worker): region files created ok region = 43cab87f-da7f-46b7-be54-ca199098e92c
112026-08-29T21:49:46.465ZINFOcrucible-agent (datafile): region 43cab87f-da7f-46b7-be54-ca199098e92c state: Requested -> Created
122026-08-29T21:49:46.465ZINFOcrucible-agent (worker): applying SMF actions post create...
132026-08-29T21:49:46.466ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-43cab87f-da7f-46b7-be54-ca199098e92c
142026-08-29T21:49:46.477ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-43cab87f-da7f-46b7-be54-ca199098e92c
152026-08-29T21:49:46.489ZINFOcrucible-agent (worker): creating config property group
162026-08-29T21:49:46.492ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-43cab87f-da7f-46b7-be54-ca199098e92c
172026-08-29T21:49:46.492ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/43cab87f-da7f-46b7-be54-ca199098e92c
182026-08-29T21:49:46.493ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192026-08-29T21:49:46.493ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
202026-08-29T21:49:46.493ZINFOcrucible-agent (worker): commit
212026-08-29T21:49:46.496ZINFOcrucible-agent (worker): ok!
222026-08-29T21:49:46.502ZINFOcrucible-agent (worker): SMF ok!
232026-08-29T21:49:46.939ZINFOcrucible-agent (dropshot): request completed latency_us = 79 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:59418 req_id = 3de0fbde-7e2a-47ac-9b67-e4d0e5976e58 response_code = 200 uri = /crucible/0/regions
242026-08-29T21:49:47.968ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:59418 req_id = 2c73c35e-52f4-4072-b705-aa67c66ff08f response_code = 200 uri = /crucible/0/regions/b7f9337b-ca83-4825-8f19-423590adbb21
252026-08-29T21:49:47.973ZINFOcrucible-agent (datafile): path is "/data/regions/b7f9337b-ca83-4825-8f19-423590adbb21"
262026-08-29T21:49:47.979ZINFOcrucible-agent (datafile): dataset is oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/b7f9337b-ca83-4825-8f19-423590adbb21
272026-08-29T21:49:47.985ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/b7f9337b-ca83-4825-8f19-423590adbb21@ede9e6c4-4093-43b3-ab68-5003b05f85fe
282026-08-29T21:49:47.985ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/b7f9337b-ca83-4825-8f19-423590adbb21", "ede9e6c4-4093-43b3-ab68-5003b05f85fe"]
292026-08-29T21:49:47.997ZINFOcrucible-agent (dropshot): request completed latency_us = 24882 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:59418 req_id = acaf6f6f-c664-4b73-87a9-4c07cc6b1c08 response_code = 200 uri = /crucible/0/regions/b7f9337b-ca83-4825-8f19-423590adbb21/snapshots/ede9e6c4-4093-43b3-ab68-5003b05f85fe
302026-08-29T21:49:48.001ZINFOcrucible-agent (datafile): path is "/data/regions/b7f9337b-ca83-4825-8f19-423590adbb21"
312026-08-29T21:49:48.008ZINFOcrucible-agent (datafile): dataset is oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/b7f9337b-ca83-4825-8f19-423590adbb21
322026-08-29T21:49:48.014ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/b7f9337b-ca83-4825-8f19-423590adbb21@ede9e6c4-4093-43b3-ab68-5003b05f85fe
332026-08-29T21:49:48.014ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/b7f9337b-ca83-4825-8f19-423590adbb21", "ede9e6c4-4093-43b3-ab68-5003b05f85fe"]
342026-08-29T21:49:48.021ZINFOcrucible-agent (datafile): requesting running snapshot b7f9337b-ca83-4825-8f19-423590adbb21-ede9e6c4-4093-43b3-ab68-5003b05f85fe state: Requested
352026-08-29T21:49:48.025ZINFOcrucible-agent (worker): applying SMF actions for region b7f9337b-ca83-4825-8f19-423590adbb21 running snapshot ede9e6c4-4093-43b3-ab68-5003b05f85fe (state Requested)...
362026-08-29T21:49:48.025ZINFOcrucible-agent (dropshot): request completed latency_us = 24391 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:59418 req_id = b62b6852-6c55-4fea-a09d-3f26f1c09e99 response_code = 200 uri = /crucible/0/regions/b7f9337b-ca83-4825-8f19-423590adbb21/snapshots/ede9e6c4-4093-43b3-ab68-5003b05f85fe/run
372026-08-29T21:49:48.028ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-b7f9337b-ca83-4825-8f19-423590adbb21-ede9e6c4-4093-43b3-ab68-5003b05f85fe
382026-08-29T21:49:48.031ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-b7f9337b-ca83-4825-8f19-423590adbb21-ede9e6c4-4093-43b3-ab68-5003b05f85fe
392026-08-29T21:49:48.043ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-b7f9337b-ca83-4825-8f19-423590adbb21-ede9e6c4-4093-43b3-ab68-5003b05f85fe
402026-08-29T21:49:48.043ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b7f9337b-ca83-4825-8f19-423590adbb21/.zfs/snapshot/ede9e6c4-4093-43b3-ab68-5003b05f85fe
412026-08-29T21:49:48.043ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
422026-08-29T21:49:48.043ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
432026-08-29T21:49:48.043ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
442026-08-29T21:49:48.043ZINFOcrucible-agent (worker): commit
452026-08-29T21:49:48.047ZINFOcrucible-agent (worker): ok!
462026-08-29T21:49:48.052ZINFOcrucible-agent (worker): SMF ok!
472026-08-29T21:49:48.052ZINFOcrucible-agent (datafile): region b7f9337b-ca83-4825-8f19-423590adbb21 running snapshot ede9e6c4-4093-43b3-ab68-5003b05f85fe state: Requested -> Created
482026-08-29T21:49:48.280ZINFOcrucible-agent (datafile): path is "/data/regions/b7f9337b-ca83-4825-8f19-423590adbb21"
492026-08-29T21:49:48.288ZINFOcrucible-agent (datafile): dataset is oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/b7f9337b-ca83-4825-8f19-423590adbb21
502026-08-29T21:49:48.294ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/b7f9337b-ca83-4825-8f19-423590adbb21@ede9e6c4-4093-43b3-ab68-5003b05f85fe
512026-08-29T21:49:48.294ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3883d706-5670-490a-b3cd-441531713e44/crucible/regions/b7f9337b-ca83-4825-8f19-423590adbb21", "ede9e6c4-4093-43b3-ab68-5003b05f85fe"]
522026-08-29T21:49:48.301ZINFOcrucible-agent (dropshot): request completed latency_us = 21112 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:59418 req_id = 99e37a1b-4e7d-4c5c-8112-ad69e094ce40 response_code = 200 uri = /crucible/0/regions/b7f9337b-ca83-4825-8f19-423590adbb21/snapshots/ede9e6c4-4093-43b3-ab68-5003b05f85fe/run