Max level shown:
12026-09-03T02:24:25.614ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::c]:52622
22026-09-03T02:24:25.615ZINFOcrucible-agent (datafile): region f0d716d0-1a8f-4867-a211-9ccb1d3fad6c state: Requested
32026-09-03T02:24:25.616ZINFOcrucible-agent (dropshot): request completed latency_us = 1244 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:52622 req_id = 3f3bf479-02a3-4247-9f4c-6ef552708f1b response_code = 200 uri = /crucible/0/regions
42026-09-03T02:24:25.616ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-09-03T02:24:25.625ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/f0d716d0-1a8f-4867-a211-9ccb1d3fad6c
62026-09-03T02:24:25.625ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/f0d716d0-1a8f-4867-a211-9ccb1d3fad6c
72026-09-03T02:24:25.671ZINFOcrucible-agent (worker): creating region Region { id: RegionId("f0d716d0-1a8f-4867-a211-9ccb1d3fad6c"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/f0d716d0-1a8f-4867-a211-9ccb1d3fad6c" region = f0d716d0-1a8f-4867-a211-9ccb1d3fad6c
82026-09-03T02:24:25.671ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "f0d716d0-1a8f-4867-a211-9ccb1d3fad6c" "--data" "/data/regions/f0d716d0-1a8f-4867-a211-9ccb1d3fad6c" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = f0d716d0-1a8f-4867-a211-9ccb1d3fad6c
92026-09-03T02:24:25.719ZINFOcrucible-agent (worker): region files created ok region = f0d716d0-1a8f-4867-a211-9ccb1d3fad6c
102026-09-03T02:24:25.719ZINFOcrucible-agent (datafile): region f0d716d0-1a8f-4867-a211-9ccb1d3fad6c state: Requested -> Created
112026-09-03T02:24:25.719ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-09-03T02:24:25.726ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-f0d716d0-1a8f-4867-a211-9ccb1d3fad6c
132026-09-03T02:24:25.728ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-f0d716d0-1a8f-4867-a211-9ccb1d3fad6c
142026-09-03T02:24:25.733ZINFOcrucible-agent (worker): creating config property group
152026-09-03T02:24:25.735ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-f0d716d0-1a8f-4867-a211-9ccb1d3fad6c
162026-09-03T02:24:25.735ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f0d716d0-1a8f-4867-a211-9ccb1d3fad6c
172026-09-03T02:24:25.735ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-09-03T02:24:25.735ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
192026-09-03T02:24:25.735ZINFOcrucible-agent (worker): commit
202026-09-03T02:24:25.737ZINFOcrucible-agent (worker): ok!
212026-09-03T02:24:25.740ZINFOcrucible-agent (worker): SMF ok!
222026-09-03T02:24:25.876ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:52622 req_id = bbe9d9f5-6b2e-403e-88cd-0f55805f6947 response_code = 200 uri = /crucible/0/regions
232026-09-03T02:24:27.346ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:52622 req_id = 6bd71846-602e-4222-b8e8-ef87d7746a01 response_code = 200 uri = /crucible/0/regions/a9351160-5bf3-4c72-9be2-085f421c0176
242026-09-03T02:24:27.350ZINFOcrucible-agent (datafile): path is "/data/regions/a9351160-5bf3-4c72-9be2-085f421c0176"
252026-09-03T02:24:27.356ZINFOcrucible-agent (datafile): dataset is oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/a9351160-5bf3-4c72-9be2-085f421c0176
262026-09-03T02:24:27.362ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/a9351160-5bf3-4c72-9be2-085f421c0176@0ed0df05-670b-43d8-87d9-eb8a7cf618b2
272026-09-03T02:24:27.363ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/a9351160-5bf3-4c72-9be2-085f421c0176", "0ed0df05-670b-43d8-87d9-eb8a7cf618b2"]
282026-09-03T02:24:27.374ZINFOcrucible-agent (dropshot): request completed latency_us = 23476 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:52622 req_id = 15cd4b04-3742-46e5-b7b8-ba181e9e5bf2 response_code = 200 uri = /crucible/0/regions/a9351160-5bf3-4c72-9be2-085f421c0176/snapshots/0ed0df05-670b-43d8-87d9-eb8a7cf618b2
292026-09-03T02:24:27.377ZINFOcrucible-agent (datafile): path is "/data/regions/a9351160-5bf3-4c72-9be2-085f421c0176"
302026-09-03T02:24:27.383ZINFOcrucible-agent (datafile): dataset is oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/a9351160-5bf3-4c72-9be2-085f421c0176
312026-09-03T02:24:27.388ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/a9351160-5bf3-4c72-9be2-085f421c0176@0ed0df05-670b-43d8-87d9-eb8a7cf618b2
322026-09-03T02:24:27.388ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/a9351160-5bf3-4c72-9be2-085f421c0176", "0ed0df05-670b-43d8-87d9-eb8a7cf618b2"]
332026-09-03T02:24:27.395ZINFOcrucible-agent (datafile): requesting running snapshot a9351160-5bf3-4c72-9be2-085f421c0176-0ed0df05-670b-43d8-87d9-eb8a7cf618b2 state: Requested
342026-09-03T02:24:27.400ZINFOcrucible-agent (worker): applying SMF actions for region a9351160-5bf3-4c72-9be2-085f421c0176 running snapshot 0ed0df05-670b-43d8-87d9-eb8a7cf618b2 (state Requested)...
352026-09-03T02:24:27.400ZINFOcrucible-agent (dropshot): request completed latency_us = 22980 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:52622 req_id = d9645172-e7ab-429a-87b8-142937041b55 response_code = 200 uri = /crucible/0/regions/a9351160-5bf3-4c72-9be2-085f421c0176/snapshots/0ed0df05-670b-43d8-87d9-eb8a7cf618b2/run
362026-09-03T02:24:27.402ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-a9351160-5bf3-4c72-9be2-085f421c0176-0ed0df05-670b-43d8-87d9-eb8a7cf618b2
372026-09-03T02:24:27.405ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-a9351160-5bf3-4c72-9be2-085f421c0176-0ed0df05-670b-43d8-87d9-eb8a7cf618b2
382026-09-03T02:24:27.410ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-a9351160-5bf3-4c72-9be2-085f421c0176-0ed0df05-670b-43d8-87d9-eb8a7cf618b2
392026-09-03T02:24:27.410ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a9351160-5bf3-4c72-9be2-085f421c0176/.zfs/snapshot/0ed0df05-670b-43d8-87d9-eb8a7cf618b2
402026-09-03T02:24:27.410ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-09-03T02:24:27.410ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-09-03T02:24:27.411ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
432026-09-03T02:24:27.411ZINFOcrucible-agent (worker): commit
442026-09-03T02:24:27.412ZINFOcrucible-agent (worker): ok!
452026-09-03T02:24:27.415ZINFOcrucible-agent (worker): SMF ok!
462026-09-03T02:24:27.415ZINFOcrucible-agent (datafile): region a9351160-5bf3-4c72-9be2-085f421c0176 running snapshot 0ed0df05-670b-43d8-87d9-eb8a7cf618b2 state: Requested -> Created
472026-09-03T02:24:27.759ZINFOcrucible-agent (datafile): path is "/data/regions/a9351160-5bf3-4c72-9be2-085f421c0176"
482026-09-03T02:24:27.765ZINFOcrucible-agent (datafile): dataset is oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/a9351160-5bf3-4c72-9be2-085f421c0176
492026-09-03T02:24:27.771ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/a9351160-5bf3-4c72-9be2-085f421c0176@0ed0df05-670b-43d8-87d9-eb8a7cf618b2
502026-09-03T02:24:27.771ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d9024315-ab46-4ae0-9671-34d5af29e510/crucible/regions/a9351160-5bf3-4c72-9be2-085f421c0176", "0ed0df05-670b-43d8-87d9-eb8a7cf618b2"]
512026-09-03T02:24:27.776ZINFOcrucible-agent (dropshot): request completed latency_us = 17483 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:52622 req_id = 7279eeba-6cc1-4fc6-8572-f4eaac2f0301 response_code = 200 uri = /crucible/0/regions/a9351160-5bf3-4c72-9be2-085f421c0176/snapshots/0ed0df05-670b-43d8-87d9-eb8a7cf618b2/run