Max level shown:
12026-04-22T02:21:40.153ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::c]:50641
22026-04-22T02:21:40.153ZINFOcrucible-agent (datafile): region 5e9bf5b1-c83e-438f-bd8e-56b9cf01e174 state: Requested
32026-04-22T02:21:40.153ZINFOcrucible-agent (dropshot): request completed latency_us = 570 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50641 req_id = 999a2bd4-f901-40ac-8496-f144383bb627 response_code = 200 uri = /crucible/0/regions
42026-04-22T02:21:40.153ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-04-22T02:21:40.163ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/5e9bf5b1-c83e-438f-bd8e-56b9cf01e174
62026-04-22T02:21:40.163ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/5e9bf5b1-c83e-438f-bd8e-56b9cf01e174
72026-04-22T02:21:40.223ZINFOcrucible-agent (worker): creating region Region { id: RegionId("5e9bf5b1-c83e-438f-bd8e-56b9cf01e174"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/5e9bf5b1-c83e-438f-bd8e-56b9cf01e174" region = 5e9bf5b1-c83e-438f-bd8e-56b9cf01e174
82026-04-22T02:21:40.223ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "5e9bf5b1-c83e-438f-bd8e-56b9cf01e174" "--data" "/data/regions/5e9bf5b1-c83e-438f-bd8e-56b9cf01e174" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 5e9bf5b1-c83e-438f-bd8e-56b9cf01e174
92026-04-22T02:21:40.300ZINFOcrucible-agent (worker): region files created ok region = 5e9bf5b1-c83e-438f-bd8e-56b9cf01e174
102026-04-22T02:21:40.300ZINFOcrucible-agent (datafile): region 5e9bf5b1-c83e-438f-bd8e-56b9cf01e174 state: Requested -> Created
112026-04-22T02:21:40.300ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-04-22T02:21:40.322ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-5e9bf5b1-c83e-438f-bd8e-56b9cf01e174
132026-04-22T02:21:40.333ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-5e9bf5b1-c83e-438f-bd8e-56b9cf01e174
142026-04-22T02:21:40.340ZINFOcrucible-agent (worker): creating config property group
152026-04-22T02:21:40.343ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-5e9bf5b1-c83e-438f-bd8e-56b9cf01e174
162026-04-22T02:21:40.343ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5e9bf5b1-c83e-438f-bd8e-56b9cf01e174
172026-04-22T02:21:40.343ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-04-22T02:21:40.343ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
192026-04-22T02:21:40.343ZINFOcrucible-agent (worker): commit
202026-04-22T02:21:40.348ZINFOcrucible-agent (worker): ok!
212026-04-22T02:21:40.352ZINFOcrucible-agent (worker): SMF ok!
222026-04-22T02:21:40.449ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50641 req_id = a1acbe58-7ab7-44bb-b447-e4d6b06431bd response_code = 200 uri = /crucible/0/regions
232026-04-22T02:21:41.886ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:50641 req_id = 72c00c88-52bb-446d-8df5-3b96afeb2806 response_code = 200 uri = /crucible/0/regions/00672394-32d4-4080-a33d-f49f9f1055a5
242026-04-22T02:21:41.893ZINFOcrucible-agent (datafile): path is "/data/regions/00672394-32d4-4080-a33d-f49f9f1055a5"
252026-04-22T02:21:41.899ZINFOcrucible-agent (datafile): dataset is oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/00672394-32d4-4080-a33d-f49f9f1055a5
262026-04-22T02:21:41.905ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/00672394-32d4-4080-a33d-f49f9f1055a5@e2e83469-9675-47ec-831f-eefd1f91750c
272026-04-22T02:21:41.905ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/00672394-32d4-4080-a33d-f49f9f1055a5", "e2e83469-9675-47ec-831f-eefd1f91750c"]
282026-04-22T02:21:41.914ZINFOcrucible-agent (dropshot): request completed latency_us = 21265 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:50641 req_id = 2bd938b8-b964-4437-8037-3d720561c1f2 response_code = 200 uri = /crucible/0/regions/00672394-32d4-4080-a33d-f49f9f1055a5/snapshots/e2e83469-9675-47ec-831f-eefd1f91750c
292026-04-22T02:21:41.917ZINFOcrucible-agent (datafile): path is "/data/regions/00672394-32d4-4080-a33d-f49f9f1055a5"
302026-04-22T02:21:41.923ZINFOcrucible-agent (datafile): dataset is oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/00672394-32d4-4080-a33d-f49f9f1055a5
312026-04-22T02:21:41.929ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/00672394-32d4-4080-a33d-f49f9f1055a5@e2e83469-9675-47ec-831f-eefd1f91750c
322026-04-22T02:21:41.929ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/00672394-32d4-4080-a33d-f49f9f1055a5", "e2e83469-9675-47ec-831f-eefd1f91750c"]
332026-04-22T02:21:41.937ZINFOcrucible-agent (datafile): requesting running snapshot 00672394-32d4-4080-a33d-f49f9f1055a5-e2e83469-9675-47ec-831f-eefd1f91750c state: Requested
342026-04-22T02:21:41.937ZINFOcrucible-agent (dropshot): request completed latency_us = 20254 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50641 req_id = ac4e553f-8609-447a-91e7-11f41a1bbe0a response_code = 200 uri = /crucible/0/regions/00672394-32d4-4080-a33d-f49f9f1055a5/snapshots/e2e83469-9675-47ec-831f-eefd1f91750c/run
352026-04-22T02:21:41.937ZINFOcrucible-agent (worker): applying SMF actions for region 00672394-32d4-4080-a33d-f49f9f1055a5 running snapshot e2e83469-9675-47ec-831f-eefd1f91750c (state Requested)...
362026-04-22T02:21:41.943ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-00672394-32d4-4080-a33d-f49f9f1055a5-e2e83469-9675-47ec-831f-eefd1f91750c
372026-04-22T02:21:41.946ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-00672394-32d4-4080-a33d-f49f9f1055a5-e2e83469-9675-47ec-831f-eefd1f91750c
382026-04-22T02:21:41.956ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-00672394-32d4-4080-a33d-f49f9f1055a5-e2e83469-9675-47ec-831f-eefd1f91750c
392026-04-22T02:21:41.956ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/00672394-32d4-4080-a33d-f49f9f1055a5/.zfs/snapshot/e2e83469-9675-47ec-831f-eefd1f91750c
402026-04-22T02:21:41.956ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-04-22T02:21:41.956ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-04-22T02:21:41.956ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432026-04-22T02:21:41.956ZINFOcrucible-agent (worker): commit
442026-04-22T02:21:41.960ZINFOcrucible-agent (worker): ok!
452026-04-22T02:21:41.964ZINFOcrucible-agent (worker): SMF ok!
462026-04-22T02:21:41.964ZINFOcrucible-agent (datafile): region 00672394-32d4-4080-a33d-f49f9f1055a5 running snapshot e2e83469-9675-47ec-831f-eefd1f91750c state: Requested -> Created
472026-04-22T02:21:42.126ZINFOcrucible-agent (datafile): path is "/data/regions/00672394-32d4-4080-a33d-f49f9f1055a5"
482026-04-22T02:21:42.132ZINFOcrucible-agent (datafile): dataset is oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/00672394-32d4-4080-a33d-f49f9f1055a5
492026-04-22T02:21:42.139ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/00672394-32d4-4080-a33d-f49f9f1055a5@e2e83469-9675-47ec-831f-eefd1f91750c
502026-04-22T02:21:42.139ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0262f759-bb9f-4afb-9585-0c3058075cc5/crucible/regions/00672394-32d4-4080-a33d-f49f9f1055a5", "e2e83469-9675-47ec-831f-eefd1f91750c"]
512026-04-22T02:21:42.145ZINFOcrucible-agent (dropshot): request completed latency_us = 19399 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:50641 req_id = 5fcffd4f-02bf-4a40-82a1-8bacd6ab524d response_code = 200 uri = /crucible/0/regions/00672394-32d4-4080-a33d-f49f9f1055a5/snapshots/e2e83469-9675-47ec-831f-eefd1f91750c/run