Max level shown:
12025-02-19T06:05:54.555ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:64465
22025-02-19T06:05:54.556ZINFOcrucible-agent (datafile): region 8076fd0a-a27b-4c72-856e-324e6b0f09f2 state: Requested
32025-02-19T06:05:54.556ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-02-19T06:05:54.556ZINFOcrucible-agent (dropshot): request completed latency_us = 692 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64465 req_id = fcb3c074-f118-4a18-9c56-cd18aac71690 response_code = 200 uri = /crucible/0/regions
52025-02-19T06:05:54.570ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/8076fd0a-a27b-4c72-856e-324e6b0f09f2
62025-02-19T06:05:54.570ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/8076fd0a-a27b-4c72-856e-324e6b0f09f2
72025-02-19T06:05:54.639ZINFOcrucible-agent (worker): creating region Region { id: RegionId("8076fd0a-a27b-4c72-856e-324e6b0f09f2"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/8076fd0a-a27b-4c72-856e-324e6b0f09f2" region = 8076fd0a-a27b-4c72-856e-324e6b0f09f2
82025-02-19T06:05:54.639ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "8076fd0a-a27b-4c72-856e-324e6b0f09f2" "--data" "/data/regions/8076fd0a-a27b-4c72-856e-324e6b0f09f2" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 8076fd0a-a27b-4c72-856e-324e6b0f09f2
92025-02-19T06:05:54.758ZINFOcrucible-agent (worker): region files created ok region = 8076fd0a-a27b-4c72-856e-324e6b0f09f2
102025-02-19T06:05:54.758ZINFOcrucible-agent (datafile): region 8076fd0a-a27b-4c72-856e-324e6b0f09f2 state: Requested -> Created
112025-02-19T06:05:54.758ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-02-19T06:05:54.759ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-8076fd0a-a27b-4c72-856e-324e6b0f09f2
132025-02-19T06:05:54.772ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-8076fd0a-a27b-4c72-856e-324e6b0f09f2
142025-02-19T06:05:54.789ZINFOcrucible-agent (worker): creating config property group
152025-02-19T06:05:54.792ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8076fd0a-a27b-4c72-856e-324e6b0f09f2
162025-02-19T06:05:54.792ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8076fd0a-a27b-4c72-856e-324e6b0f09f2
172025-02-19T06:05:54.792ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-02-19T06:05:54.792ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
192025-02-19T06:05:54.793ZINFOcrucible-agent (worker): commit
202025-02-19T06:05:54.797ZINFOcrucible-agent (worker): ok!
212025-02-19T06:05:54.803ZINFOcrucible-agent (worker): SMF ok!
222025-02-19T06:05:54.879ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64465 req_id = 0f64eaa9-1e81-4b4a-a7be-0712195bef8a response_code = 200 uri = /crucible/0/regions
232025-02-19T06:05:56.197ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:64465 req_id = 75d5828c-b0f3-41f6-a641-d0c9f8e0ee78 response_code = 200 uri = /crucible/0/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95
242025-02-19T06:05:56.201ZINFOcrucible-agent (datafile): path is "/data/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95"
252025-02-19T06:05:56.210ZINFOcrucible-agent (datafile): dataset is oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95
262025-02-19T06:05:56.218ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95@0155c35f-5786-469d-98d7-5a2d3b1980a6
272025-02-19T06:05:56.218ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95", "0155c35f-5786-469d-98d7-5a2d3b1980a6"]
282025-02-19T06:05:56.230ZINFOcrucible-agent (dropshot): request completed latency_us = 29532 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:64465 req_id = 3151a567-e9fb-4094-a465-1b4bbf3e3006 response_code = 200 uri = /crucible/0/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95/snapshots/0155c35f-5786-469d-98d7-5a2d3b1980a6
292025-02-19T06:05:56.234ZINFOcrucible-agent (datafile): path is "/data/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95"
302025-02-19T06:05:56.244ZINFOcrucible-agent (datafile): dataset is oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95
312025-02-19T06:05:56.252ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95@0155c35f-5786-469d-98d7-5a2d3b1980a6
322025-02-19T06:05:56.252ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95", "0155c35f-5786-469d-98d7-5a2d3b1980a6"]
332025-02-19T06:05:56.262ZINFOcrucible-agent (datafile): requesting running snapshot ffa93da4-e07b-41b7-8734-e07c9edb9b95-0155c35f-5786-469d-98d7-5a2d3b1980a6 state: Requested
342025-02-19T06:05:56.262ZINFOcrucible-agent (worker): applying SMF actions for region ffa93da4-e07b-41b7-8734-e07c9edb9b95 running snapshot 0155c35f-5786-469d-98d7-5a2d3b1980a6 (state Requested)...
352025-02-19T06:05:56.262ZINFOcrucible-agent (dropshot): request completed latency_us = 28212 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64465 req_id = e2f0de3c-f8b5-44bc-9757-9296d92bc8bb response_code = 200 uri = /crucible/0/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95/snapshots/0155c35f-5786-469d-98d7-5a2d3b1980a6/run
362025-02-19T06:05:56.265ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-ffa93da4-e07b-41b7-8734-e07c9edb9b95-0155c35f-5786-469d-98d7-5a2d3b1980a6
372025-02-19T06:05:56.268ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-ffa93da4-e07b-41b7-8734-e07c9edb9b95-0155c35f-5786-469d-98d7-5a2d3b1980a6
382025-02-19T06:05:56.277ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-ffa93da4-e07b-41b7-8734-e07c9edb9b95-0155c35f-5786-469d-98d7-5a2d3b1980a6
392025-02-19T06:05:56.277ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95/.zfs/snapshot/0155c35f-5786-469d-98d7-5a2d3b1980a6
402025-02-19T06:05:56.277ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-02-19T06:05:56.278ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-02-19T06:05:56.278ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432025-02-19T06:05:56.278ZINFOcrucible-agent (worker): commit
442025-02-19T06:05:56.281ZINFOcrucible-agent (worker): ok!
452025-02-19T06:05:56.285ZINFOcrucible-agent (worker): SMF ok!
462025-02-19T06:05:56.285ZINFOcrucible-agent (datafile): region ffa93da4-e07b-41b7-8734-e07c9edb9b95 running snapshot 0155c35f-5786-469d-98d7-5a2d3b1980a6 state: Requested -> Created
472025-02-19T06:05:56.547ZINFOcrucible-agent (datafile): path is "/data/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95"
482025-02-19T06:05:56.556ZINFOcrucible-agent (datafile): dataset is oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95
492025-02-19T06:05:56.565ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95@0155c35f-5786-469d-98d7-5a2d3b1980a6
502025-02-19T06:05:56.565ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0fe01249-d314-460c-80d9-930d53a97dda/crucible/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95", "0155c35f-5786-469d-98d7-5a2d3b1980a6"]
512025-02-19T06:05:56.574ZINFOcrucible-agent (dropshot): request completed latency_us = 26447 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:64465 req_id = 1d9c469d-7e94-4f1e-94c5-204890cecf8e response_code = 200 uri = /crucible/0/regions/ffa93da4-e07b-41b7-8734-e07c9edb9b95/snapshots/0155c35f-5786-469d-98d7-5a2d3b1980a6/run