Max level shown:
12026-08-21T18:50:49.066ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:34548
22026-08-21T18:50:49.067ZINFOcrucible-agent (datafile): region ece7863f-1579-4a79-8ba2-f1677ec55d55 state: Requested
32026-08-21T18:50:49.067ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-08-21T18:50:49.067ZINFOcrucible-agent (dropshot): request completed latency_us = 739 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34548 req_id = 1d4303e6-bf9e-4d89-99eb-c424ba9c5760 response_code = 200 uri = /crucible/0/regions
52026-08-21T18:50:49.077ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/ece7863f-1579-4a79-8ba2-f1677ec55d55
62026-08-21T18:50:49.077ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/ece7863f-1579-4a79-8ba2-f1677ec55d55
72026-08-21T18:50:49.129ZINFOcrucible-agent (worker): creating region Region { id: RegionId("ece7863f-1579-4a79-8ba2-f1677ec55d55"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/ece7863f-1579-4a79-8ba2-f1677ec55d55" region = ece7863f-1579-4a79-8ba2-f1677ec55d55
82026-08-21T18:50:49.129ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "ece7863f-1579-4a79-8ba2-f1677ec55d55" "--data" "/data/regions/ece7863f-1579-4a79-8ba2-f1677ec55d55" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = ece7863f-1579-4a79-8ba2-f1677ec55d55
92026-08-21T18:50:49.175ZINFOcrucible-agent (worker): region files created ok region = ece7863f-1579-4a79-8ba2-f1677ec55d55
102026-08-21T18:50:49.175ZINFOcrucible-agent (datafile): region ece7863f-1579-4a79-8ba2-f1677ec55d55 state: Requested -> Created
112026-08-21T18:50:49.175ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-08-21T18:50:49.189ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-ece7863f-1579-4a79-8ba2-f1677ec55d55
132026-08-21T18:50:49.201ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-ece7863f-1579-4a79-8ba2-f1677ec55d55
142026-08-21T18:50:49.207ZINFOcrucible-agent (worker): creating config property group
152026-08-21T18:50:49.209ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-ece7863f-1579-4a79-8ba2-f1677ec55d55
162026-08-21T18:50:49.209ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ece7863f-1579-4a79-8ba2-f1677ec55d55
172026-08-21T18:50:49.209ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-08-21T18:50:49.209ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
192026-08-21T18:50:49.209ZINFOcrucible-agent (worker): commit
202026-08-21T18:50:49.211ZINFOcrucible-agent (worker): ok!
212026-08-21T18:50:49.213ZINFOcrucible-agent (worker): SMF ok!
222026-08-21T18:50:49.253ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34548 req_id = 5f994f1e-8d88-4b22-8745-fb48d748223d response_code = 200 uri = /crucible/0/regions
232026-08-21T18:50:51.297ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34548 req_id = 5a92d958-4688-4f5b-85a2-f24172c088d2 response_code = 200 uri = /crucible/0/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c
242026-08-21T18:50:51.300ZINFOcrucible-agent (datafile): path is "/data/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c"
252026-08-21T18:50:51.306ZINFOcrucible-agent (datafile): dataset is oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c
262026-08-21T18:50:51.312ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c@2da0fcdc-6a78-4fca-80e1-ee8213873d44
272026-08-21T18:50:51.312ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c", "2da0fcdc-6a78-4fca-80e1-ee8213873d44"]
282026-08-21T18:50:51.324ZINFOcrucible-agent (dropshot): request completed latency_us = 23907 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34548 req_id = e7e89670-c35b-4b49-9435-d8ea32eee4aa response_code = 200 uri = /crucible/0/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c/snapshots/2da0fcdc-6a78-4fca-80e1-ee8213873d44
292026-08-21T18:50:51.328ZINFOcrucible-agent (datafile): path is "/data/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c"
302026-08-21T18:50:51.333ZINFOcrucible-agent (datafile): dataset is oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c
312026-08-21T18:50:51.339ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c@2da0fcdc-6a78-4fca-80e1-ee8213873d44
322026-08-21T18:50:51.339ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c", "2da0fcdc-6a78-4fca-80e1-ee8213873d44"]
332026-08-21T18:50:51.346ZINFOcrucible-agent (datafile): requesting running snapshot 2670e382-17c7-4fa6-ad94-eaeee148b67c-2da0fcdc-6a78-4fca-80e1-ee8213873d44 state: Requested
342026-08-21T18:50:51.350ZINFOcrucible-agent (worker): applying SMF actions for region 2670e382-17c7-4fa6-ad94-eaeee148b67c running snapshot 2da0fcdc-6a78-4fca-80e1-ee8213873d44 (state Requested)...
352026-08-21T18:50:51.350ZINFOcrucible-agent (dropshot): request completed latency_us = 22420 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34548 req_id = 16d9a531-927f-4441-a713-f13dd131cbe7 response_code = 200 uri = /crucible/0/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c/snapshots/2da0fcdc-6a78-4fca-80e1-ee8213873d44/run
362026-08-21T18:50:51.353ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-2670e382-17c7-4fa6-ad94-eaeee148b67c-2da0fcdc-6a78-4fca-80e1-ee8213873d44
372026-08-21T18:50:51.354ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-2670e382-17c7-4fa6-ad94-eaeee148b67c-2da0fcdc-6a78-4fca-80e1-ee8213873d44
382026-08-21T18:50:51.359ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-2670e382-17c7-4fa6-ad94-eaeee148b67c-2da0fcdc-6a78-4fca-80e1-ee8213873d44
392026-08-21T18:50:51.360ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c/.zfs/snapshot/2da0fcdc-6a78-4fca-80e1-ee8213873d44
402026-08-21T18:50:51.360ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-08-21T18:50:51.360ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-08-21T18:50:51.360ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432026-08-21T18:50:51.360ZINFOcrucible-agent (worker): commit
442026-08-21T18:50:51.362ZINFOcrucible-agent (worker): ok!
452026-08-21T18:50:51.364ZINFOcrucible-agent (worker): SMF ok!
462026-08-21T18:50:51.364ZINFOcrucible-agent (datafile): region 2670e382-17c7-4fa6-ad94-eaeee148b67c running snapshot 2da0fcdc-6a78-4fca-80e1-ee8213873d44 state: Requested -> Created
472026-08-21T18:50:51.564ZINFOcrucible-agent (datafile): path is "/data/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c"
482026-08-21T18:50:51.570ZINFOcrucible-agent (datafile): dataset is oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c
492026-08-21T18:50:51.576ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c@2da0fcdc-6a78-4fca-80e1-ee8213873d44
502026-08-21T18:50:51.576ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_2e4ed4af-7169-42d4-a2f9-c668ef0f39ea/crucible/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c", "2da0fcdc-6a78-4fca-80e1-ee8213873d44"]
512026-08-21T18:50:51.582ZINFOcrucible-agent (dropshot): request completed latency_us = 18559 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34548 req_id = a57fa5f3-41de-4fd7-8bc1-2da2cdead0e3 response_code = 200 uri = /crucible/0/regions/2670e382-17c7-4fa6-ad94-eaeee148b67c/snapshots/2da0fcdc-6a78-4fca-80e1-ee8213873d44/run