Max level shown:
12025-12-23T01:49:00.211ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:37814
22025-12-23T01:49:00.211ZINFOcrucible-agent (datafile): region 565f3f64-68c3-4474-ac7b-d1b604d0620d state: Requested
32025-12-23T01:49:00.212ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-12-23T01:49:00.212ZINFOcrucible-agent (dropshot): request completed latency_us = 1045 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:37814 req_id = fba45aae-a198-4bbe-acc6-97bc2b166c92 response_code = 200 uri = /crucible/0/regions
52025-12-23T01:49:00.220ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/565f3f64-68c3-4474-ac7b-d1b604d0620d
62025-12-23T01:49:00.220ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/565f3f64-68c3-4474-ac7b-d1b604d0620d
72025-12-23T01:49:00.266ZINFOcrucible-agent (worker): creating region Region { id: RegionId("565f3f64-68c3-4474-ac7b-d1b604d0620d"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/565f3f64-68c3-4474-ac7b-d1b604d0620d" region = 565f3f64-68c3-4474-ac7b-d1b604d0620d
82025-12-23T01:49:00.266ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "565f3f64-68c3-4474-ac7b-d1b604d0620d" "--data" "/data/regions/565f3f64-68c3-4474-ac7b-d1b604d0620d" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 565f3f64-68c3-4474-ac7b-d1b604d0620d
92025-12-23T01:49:00.345ZINFOcrucible-agent (worker): region files created ok region = 565f3f64-68c3-4474-ac7b-d1b604d0620d
102025-12-23T01:49:00.345ZINFOcrucible-agent (datafile): region 565f3f64-68c3-4474-ac7b-d1b604d0620d state: Requested -> Created
112025-12-23T01:49:00.345ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-12-23T01:49:00.346ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-565f3f64-68c3-4474-ac7b-d1b604d0620d
132025-12-23T01:49:00.363ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-565f3f64-68c3-4474-ac7b-d1b604d0620d
142025-12-23T01:49:00.382ZINFOcrucible-agent (worker): creating config property group
152025-12-23T01:49:00.384ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:37814 req_id = 93ec38ce-ef00-450c-b900-aed1b0323226 response_code = 200 uri = /crucible/0/regions
162025-12-23T01:49:00.386ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-565f3f64-68c3-4474-ac7b-d1b604d0620d
172025-12-23T01:49:00.386ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/565f3f64-68c3-4474-ac7b-d1b604d0620d
182025-12-23T01:49:00.386ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
192025-12-23T01:49:00.386ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
202025-12-23T01:49:00.386ZINFOcrucible-agent (worker): commit
212025-12-23T01:49:00.390ZINFOcrucible-agent (worker): ok!
222025-12-23T01:49:00.409ZINFOcrucible-agent (worker): SMF ok!
232025-12-23T01:49:01.186ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:37814 req_id = 2ec89e1b-7e9d-49f6-b67a-c6c75b0ff567 response_code = 200 uri = /crucible/0/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1
242025-12-23T01:49:01.189ZINFOcrucible-agent (datafile): path is "/data/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1"
252025-12-23T01:49:01.196ZINFOcrucible-agent (datafile): dataset is oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1
262025-12-23T01:49:01.203ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1@45477e06-0bdf-482e-973b-3eb2b7281fea
272025-12-23T01:49:01.203ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1", "45477e06-0bdf-482e-973b-3eb2b7281fea"]
282025-12-23T01:49:01.211ZINFOcrucible-agent (dropshot): request completed latency_us = 21739 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:37814 req_id = 828e7484-ec47-4c8c-9e82-15e8cfd63825 response_code = 200 uri = /crucible/0/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1/snapshots/45477e06-0bdf-482e-973b-3eb2b7281fea
292025-12-23T01:49:01.214ZINFOcrucible-agent (datafile): path is "/data/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1"
302025-12-23T01:49:01.221ZINFOcrucible-agent (datafile): dataset is oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1
312025-12-23T01:49:01.226ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1@45477e06-0bdf-482e-973b-3eb2b7281fea
322025-12-23T01:49:01.226ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1", "45477e06-0bdf-482e-973b-3eb2b7281fea"]
332025-12-23T01:49:01.234ZINFOcrucible-agent (datafile): requesting running snapshot f2128925-c697-48f3-8b0e-3e5b9a784aa1-45477e06-0bdf-482e-973b-3eb2b7281fea state: Requested
342025-12-23T01:49:01.234ZINFOcrucible-agent (worker): applying SMF actions for region f2128925-c697-48f3-8b0e-3e5b9a784aa1 running snapshot 45477e06-0bdf-482e-973b-3eb2b7281fea (state Requested)...
352025-12-23T01:49:01.234ZINFOcrucible-agent (dropshot): request completed latency_us = 19807 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:37814 req_id = cec38e55-16c6-445b-a2d0-8822c633299c response_code = 200 uri = /crucible/0/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1/snapshots/45477e06-0bdf-482e-973b-3eb2b7281fea/run
362025-12-23T01:49:01.239ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-f2128925-c697-48f3-8b0e-3e5b9a784aa1-45477e06-0bdf-482e-973b-3eb2b7281fea
372025-12-23T01:49:01.242ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-f2128925-c697-48f3-8b0e-3e5b9a784aa1-45477e06-0bdf-482e-973b-3eb2b7281fea
382025-12-23T01:49:01.252ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-f2128925-c697-48f3-8b0e-3e5b9a784aa1-45477e06-0bdf-482e-973b-3eb2b7281fea
392025-12-23T01:49:01.252ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1/.zfs/snapshot/45477e06-0bdf-482e-973b-3eb2b7281fea
402025-12-23T01:49:01.252ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-12-23T01:49:01.252ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-12-23T01:49:01.252ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432025-12-23T01:49:01.252ZINFOcrucible-agent (worker): commit
442025-12-23T01:49:01.255ZINFOcrucible-agent (worker): ok!
452025-12-23T01:49:01.259ZINFOcrucible-agent (worker): SMF ok!
462025-12-23T01:49:01.259ZINFOcrucible-agent (datafile): region f2128925-c697-48f3-8b0e-3e5b9a784aa1 running snapshot 45477e06-0bdf-482e-973b-3eb2b7281fea state: Requested -> Created
472025-12-23T01:49:01.408ZINFOcrucible-agent (datafile): path is "/data/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1"
482025-12-23T01:49:01.415ZINFOcrucible-agent (datafile): dataset is oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1
492025-12-23T01:49:01.422ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1@45477e06-0bdf-482e-973b-3eb2b7281fea
502025-12-23T01:49:01.422ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b72b3fd7-3269-434b-88fe-93809c0271cf/crucible/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1", "45477e06-0bdf-482e-973b-3eb2b7281fea"]
512025-12-23T01:49:01.430ZINFOcrucible-agent (dropshot): request completed latency_us = 21301 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:37814 req_id = 856c4f43-0e74-4648-8f91-2b76b3d23a24 response_code = 200 uri = /crucible/0/regions/f2128925-c697-48f3-8b0e-3e5b9a784aa1/snapshots/45477e06-0bdf-482e-973b-3eb2b7281fea/run