Max level shown:
12026-05-06T02:04:11.765ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::c]:42716
22026-05-06T02:04:11.765ZINFOcrucible-agent (datafile): region 34b84c90-72f7-418d-b711-1802914f457c state: Requested
32026-05-06T02:04:11.766ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-05-06T02:04:11.766ZINFOcrucible-agent (dropshot): request completed latency_us = 1040 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42716 req_id = fb6a4b75-1d75-4077-8f37-2c9061dfdeb5 response_code = 200 uri = /crucible/0/regions
52026-05-06T02:04:11.777ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/34b84c90-72f7-418d-b711-1802914f457c
62026-05-06T02:04:11.777ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/34b84c90-72f7-418d-b711-1802914f457c
72026-05-06T02:04:11.832ZINFOcrucible-agent (worker): creating region Region { id: RegionId("34b84c90-72f7-418d-b711-1802914f457c"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/34b84c90-72f7-418d-b711-1802914f457c" region = 34b84c90-72f7-418d-b711-1802914f457c
82026-05-06T02:04:11.832ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "34b84c90-72f7-418d-b711-1802914f457c" "--data" "/data/regions/34b84c90-72f7-418d-b711-1802914f457c" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 34b84c90-72f7-418d-b711-1802914f457c
92026-05-06T02:04:11.907ZINFOcrucible-agent (worker): region files created ok region = 34b84c90-72f7-418d-b711-1802914f457c
102026-05-06T02:04:11.907ZINFOcrucible-agent (datafile): region 34b84c90-72f7-418d-b711-1802914f457c state: Requested -> Created
112026-05-06T02:04:11.907ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-05-06T02:04:11.926ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-34b84c90-72f7-418d-b711-1802914f457c
132026-05-06T02:04:11.931ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-34b84c90-72f7-418d-b711-1802914f457c
142026-05-06T02:04:11.941ZINFOcrucible-agent (worker): creating config property group
152026-05-06T02:04:11.944ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-34b84c90-72f7-418d-b711-1802914f457c
162026-05-06T02:04:11.944ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/34b84c90-72f7-418d-b711-1802914f457c
172026-05-06T02:04:11.944ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-05-06T02:04:11.944ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
192026-05-06T02:04:11.944ZINFOcrucible-agent (worker): commit
202026-05-06T02:04:11.949ZINFOcrucible-agent (worker): ok!
212026-05-06T02:04:11.953ZINFOcrucible-agent (worker): SMF ok!
222026-05-06T02:04:12.144ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42716 req_id = d1033622-35f4-4c67-8036-a40babaef5f8 response_code = 200 uri = /crucible/0/regions
232026-05-06T02:04:13.195ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:42716 req_id = f2f4c968-4bfc-4789-a963-665574b4a2fe response_code = 200 uri = /crucible/0/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf
242026-05-06T02:04:13.198ZINFOcrucible-agent (datafile): path is "/data/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf"
252026-05-06T02:04:13.206ZINFOcrucible-agent (datafile): dataset is oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf
262026-05-06T02:04:13.213ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf@55d8fe49-4301-4305-bf98-566a7bf27526
272026-05-06T02:04:13.213ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf", "55d8fe49-4301-4305-bf98-566a7bf27526"]
282026-05-06T02:04:13.220ZINFOcrucible-agent (dropshot): request completed latency_us = 22668 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:42716 req_id = b46e53f4-30ff-4eec-8b5f-c9e0e354bfeb response_code = 200 uri = /crucible/0/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf/snapshots/55d8fe49-4301-4305-bf98-566a7bf27526
292026-05-06T02:04:13.224ZINFOcrucible-agent (datafile): path is "/data/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf"
302026-05-06T02:04:13.232ZINFOcrucible-agent (datafile): dataset is oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf
312026-05-06T02:04:13.237ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf@55d8fe49-4301-4305-bf98-566a7bf27526
322026-05-06T02:04:13.237ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf", "55d8fe49-4301-4305-bf98-566a7bf27526"]
332026-05-06T02:04:13.246ZINFOcrucible-agent (datafile): requesting running snapshot 24137db3-5a4e-4cf5-92b0-377999fcedcf-55d8fe49-4301-4305-bf98-566a7bf27526 state: Requested
342026-05-06T02:04:13.246ZINFOcrucible-agent (dropshot): request completed latency_us = 21974 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42716 req_id = 36bb08ca-695a-42a8-9ce2-cd592b1ff19f response_code = 200 uri = /crucible/0/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf/snapshots/55d8fe49-4301-4305-bf98-566a7bf27526/run
352026-05-06T02:04:13.246ZINFOcrucible-agent (worker): applying SMF actions for region 24137db3-5a4e-4cf5-92b0-377999fcedcf running snapshot 55d8fe49-4301-4305-bf98-566a7bf27526 (state Requested)...
362026-05-06T02:04:13.251ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-24137db3-5a4e-4cf5-92b0-377999fcedcf-55d8fe49-4301-4305-bf98-566a7bf27526
372026-05-06T02:04:13.259ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-24137db3-5a4e-4cf5-92b0-377999fcedcf-55d8fe49-4301-4305-bf98-566a7bf27526
382026-05-06T02:04:13.272ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-24137db3-5a4e-4cf5-92b0-377999fcedcf-55d8fe49-4301-4305-bf98-566a7bf27526
392026-05-06T02:04:13.272ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf/.zfs/snapshot/55d8fe49-4301-4305-bf98-566a7bf27526
402026-05-06T02:04:13.272ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-05-06T02:04:13.272ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-05-06T02:04:13.272ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
432026-05-06T02:04:13.272ZINFOcrucible-agent (worker): commit
442026-05-06T02:04:13.277ZINFOcrucible-agent (worker): ok!
452026-05-06T02:04:13.281ZINFOcrucible-agent (worker): SMF ok!
462026-05-06T02:04:13.281ZINFOcrucible-agent (datafile): region 24137db3-5a4e-4cf5-92b0-377999fcedcf running snapshot 55d8fe49-4301-4305-bf98-566a7bf27526 state: Requested -> Created
472026-05-06T02:04:13.491ZINFOcrucible-agent (datafile): path is "/data/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf"
482026-05-06T02:04:13.496ZINFOcrucible-agent (datafile): dataset is oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf
492026-05-06T02:04:13.503ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf@55d8fe49-4301-4305-bf98-566a7bf27526
502026-05-06T02:04:13.503ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_1a19df70-b410-42e8-8b8f-0e79c35f6837/crucible/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf", "55d8fe49-4301-4305-bf98-566a7bf27526"]
512026-05-06T02:04:13.509ZINFOcrucible-agent (dropshot): request completed latency_us = 18136 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:42716 req_id = bdfb88de-6a0d-436c-81fb-557774ce8a50 response_code = 200 uri = /crucible/0/regions/24137db3-5a4e-4cf5-92b0-377999fcedcf/snapshots/55d8fe49-4301-4305-bf98-566a7bf27526/run