Max level shown:
12026-10-16T00:30:58.750ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::c]:40502
22026-10-16T00:30:58.750ZINFOcrucible-agent (datafile): region eb2db9dd-4920-4c4f-aa29-a7833931b487 state: Requested
32026-10-16T00:30:58.751ZINFOcrucible-agent (dropshot): request completed latency_us = 681 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:40502 req_id = e7650415-86a0-4e3a-9296-2ece49343624 response_code = 200 uri = /crucible/0/regions
42026-10-16T00:30:58.751ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-10-16T00:30:58.758ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/eb2db9dd-4920-4c4f-aa29-a7833931b487
62026-10-16T00:30:58.758ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/eb2db9dd-4920-4c4f-aa29-a7833931b487
72026-10-16T00:30:58.812ZINFOcrucible-agent (worker): creating region Region { id: RegionId("eb2db9dd-4920-4c4f-aa29-a7833931b487"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/eb2db9dd-4920-4c4f-aa29-a7833931b487" region = eb2db9dd-4920-4c4f-aa29-a7833931b487
82026-10-16T00:30:58.812ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "eb2db9dd-4920-4c4f-aa29-a7833931b487" "--data" "/data/regions/eb2db9dd-4920-4c4f-aa29-a7833931b487" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = eb2db9dd-4920-4c4f-aa29-a7833931b487
92026-10-16T00:30:58.900ZINFOcrucible-agent (worker): region files created ok region = eb2db9dd-4920-4c4f-aa29-a7833931b487
102026-10-16T00:30:58.900ZINFOcrucible-agent (datafile): region eb2db9dd-4920-4c4f-aa29-a7833931b487 state: Requested -> Created
112026-10-16T00:30:58.900ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-10-16T00:30:58.911ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-eb2db9dd-4920-4c4f-aa29-a7833931b487
132026-10-16T00:30:58.915ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-eb2db9dd-4920-4c4f-aa29-a7833931b487
142026-10-16T00:30:58.927ZINFOcrucible-agent (worker): creating config property group
152026-10-16T00:30:58.931ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-eb2db9dd-4920-4c4f-aa29-a7833931b487
162026-10-16T00:30:58.932ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/eb2db9dd-4920-4c4f-aa29-a7833931b487
172026-10-16T00:30:58.932ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-10-16T00:30:58.932ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192026-10-16T00:30:58.932ZINFOcrucible-agent (worker): commit
202026-10-16T00:30:58.936ZINFOcrucible-agent (worker): ok!
212026-10-16T00:30:58.942ZINFOcrucible-agent (worker): SMF ok!
222026-10-16T00:30:58.942ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:40502 req_id = 30b491e5-63d9-4d14-b698-763296194626 response_code = 200 uri = /crucible/0/regions
232026-10-16T00:31:00.080ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:40502 req_id = 34d23774-48a6-4f1d-aaf4-5f85dc04436d response_code = 200 uri = /crucible/0/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b
242026-10-16T00:31:00.085ZINFOcrucible-agent (datafile): path is "/data/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b"
252026-10-16T00:31:00.091ZINFOcrucible-agent (datafile): dataset is oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b
262026-10-16T00:31:00.098ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b@f9e678fe-ec25-4126-bdbb-d41614813b67
272026-10-16T00:31:00.098ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b", "f9e678fe-ec25-4126-bdbb-d41614813b67"]
282026-10-16T00:31:00.110ZINFOcrucible-agent (dropshot): request completed latency_us = 27145 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:40502 req_id = 14774511-3c67-459a-a71f-72db59087778 response_code = 200 uri = /crucible/0/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b/snapshots/f9e678fe-ec25-4126-bdbb-d41614813b67
292026-10-16T00:31:00.124ZINFOcrucible-agent (datafile): path is "/data/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b"
302026-10-16T00:31:00.131ZINFOcrucible-agent (datafile): dataset is oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b
312026-10-16T00:31:00.138ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b@f9e678fe-ec25-4126-bdbb-d41614813b67
322026-10-16T00:31:00.138ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b", "f9e678fe-ec25-4126-bdbb-d41614813b67"]
332026-10-16T00:31:00.145ZINFOcrucible-agent (datafile): requesting running snapshot 1526fdbc-1a8f-4e7d-aa8b-422620f3f49b-f9e678fe-ec25-4126-bdbb-d41614813b67 state: Requested
342026-10-16T00:31:00.145ZINFOcrucible-agent (dropshot): request completed latency_us = 21074 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:40502 req_id = dfee38f5-b200-4784-8c24-008745d221a4 response_code = 200 uri = /crucible/0/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b/snapshots/f9e678fe-ec25-4126-bdbb-d41614813b67/run
352026-10-16T00:31:00.145ZINFOcrucible-agent (worker): applying SMF actions for region 1526fdbc-1a8f-4e7d-aa8b-422620f3f49b running snapshot f9e678fe-ec25-4126-bdbb-d41614813b67 (state Requested)...
362026-10-16T00:31:00.147ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-1526fdbc-1a8f-4e7d-aa8b-422620f3f49b-f9e678fe-ec25-4126-bdbb-d41614813b67
372026-10-16T00:31:00.156ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-1526fdbc-1a8f-4e7d-aa8b-422620f3f49b-f9e678fe-ec25-4126-bdbb-d41614813b67
382026-10-16T00:31:00.170ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-1526fdbc-1a8f-4e7d-aa8b-422620f3f49b-f9e678fe-ec25-4126-bdbb-d41614813b67
392026-10-16T00:31:00.170ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b/.zfs/snapshot/f9e678fe-ec25-4126-bdbb-d41614813b67
402026-10-16T00:31:00.170ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-10-16T00:31:00.170ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-10-16T00:31:00.170ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432026-10-16T00:31:00.170ZINFOcrucible-agent (worker): commit
442026-10-16T00:31:00.175ZINFOcrucible-agent (worker): ok!
452026-10-16T00:31:00.180ZINFOcrucible-agent (worker): SMF ok!
462026-10-16T00:31:00.180ZINFOcrucible-agent (datafile): region 1526fdbc-1a8f-4e7d-aa8b-422620f3f49b running snapshot f9e678fe-ec25-4126-bdbb-d41614813b67 state: Requested -> Created
472026-10-16T00:31:00.380ZINFOcrucible-agent (datafile): path is "/data/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b"
482026-10-16T00:31:00.386ZINFOcrucible-agent (datafile): dataset is oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b
492026-10-16T00:31:00.392ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b@f9e678fe-ec25-4126-bdbb-d41614813b67
502026-10-16T00:31:00.392ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_5cce9539-e9b9-4faa-ba78-219ca80a8100/crucible/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b", "f9e678fe-ec25-4126-bdbb-d41614813b67"]
512026-10-16T00:31:00.398ZINFOcrucible-agent (dropshot): request completed latency_us = 18221 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:40502 req_id = 3c774053-9c40-4fc5-9f4d-8550c1f16053 response_code = 200 uri = /crucible/0/regions/1526fdbc-1a8f-4e7d-aa8b-422620f3f49b/snapshots/f9e678fe-ec25-4126-bdbb-d41614813b67/run