Max level shown:
12025-11-18T00:07:52.994ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:41340
22025-11-18T00:07:52.994ZINFOcrucible-agent (datafile): region bc444a66-7183-4c19-a607-0723a9b19476 state: Requested
32025-11-18T00:07:52.996ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-11-18T00:07:52.996ZINFOcrucible-agent (dropshot): request completed latency_us = 1879 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:41340 req_id = dc9a4843-ecd5-498e-ab96-de96262695df response_code = 200 uri = /crucible/0/regions
52025-11-18T00:07:53.005ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/bc444a66-7183-4c19-a607-0723a9b19476
62025-11-18T00:07:53.005ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/bc444a66-7183-4c19-a607-0723a9b19476
72025-11-18T00:07:53.081ZINFOcrucible-agent (worker): creating region Region { id: RegionId("bc444a66-7183-4c19-a607-0723a9b19476"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/bc444a66-7183-4c19-a607-0723a9b19476" region = bc444a66-7183-4c19-a607-0723a9b19476
82025-11-18T00:07:53.082ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "bc444a66-7183-4c19-a607-0723a9b19476" "--data" "/data/regions/bc444a66-7183-4c19-a607-0723a9b19476" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = bc444a66-7183-4c19-a607-0723a9b19476
92025-11-18T00:07:53.231ZINFOcrucible-agent (worker): region files created ok region = bc444a66-7183-4c19-a607-0723a9b19476
102025-11-18T00:07:53.231ZINFOcrucible-agent (datafile): region bc444a66-7183-4c19-a607-0723a9b19476 state: Requested -> Created
112025-11-18T00:07:53.231ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-11-18T00:07:53.232ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-bc444a66-7183-4c19-a607-0723a9b19476
132025-11-18T00:07:53.253ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-bc444a66-7183-4c19-a607-0723a9b19476
142025-11-18T00:07:53.293ZINFOcrucible-agent (worker): creating config property group
152025-11-18T00:07:53.303ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-bc444a66-7183-4c19-a607-0723a9b19476
162025-11-18T00:07:53.303ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/bc444a66-7183-4c19-a607-0723a9b19476
172025-11-18T00:07:53.303ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-11-18T00:07:53.303ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192025-11-18T00:07:53.303ZINFOcrucible-agent (worker): commit
202025-11-18T00:07:53.311ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:41340 req_id = dc431903-e6a7-4caa-b015-cf57341318fb response_code = 200 uri = /crucible/0/regions
212025-11-18T00:07:53.313ZINFOcrucible-agent (worker): ok!
222025-11-18T00:07:53.351ZINFOcrucible-agent (worker): SMF ok!
232025-11-18T00:07:55.188ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:41340 req_id = 4cfd7064-0cae-4efc-a1da-a29ffc41bdc3 response_code = 200 uri = /crucible/0/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1
242025-11-18T00:07:55.191ZINFOcrucible-agent (datafile): path is "/data/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1"
252025-11-18T00:07:55.197ZINFOcrucible-agent (datafile): dataset is oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1
262025-11-18T00:07:55.204ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1@302e67d8-0457-4826-8058-830e75ae370a
272025-11-18T00:07:55.204ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1", "302e67d8-0457-4826-8058-830e75ae370a"]
282025-11-18T00:07:55.215ZINFOcrucible-agent (dropshot): request completed latency_us = 23834 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:41340 req_id = a706949a-f2d4-470c-b3f4-e28e049d5276 response_code = 200 uri = /crucible/0/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1/snapshots/302e67d8-0457-4826-8058-830e75ae370a
292025-11-18T00:07:55.219ZINFOcrucible-agent (datafile): path is "/data/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1"
302025-11-18T00:07:55.225ZINFOcrucible-agent (datafile): dataset is oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1
312025-11-18T00:07:55.231ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1@302e67d8-0457-4826-8058-830e75ae370a
322025-11-18T00:07:55.231ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1", "302e67d8-0457-4826-8058-830e75ae370a"]
332025-11-18T00:07:55.241ZINFOcrucible-agent (datafile): requesting running snapshot fc9804d7-710a-4c0c-9224-dbe3d38a10e1-302e67d8-0457-4826-8058-830e75ae370a state: Requested
342025-11-18T00:07:55.241ZINFOcrucible-agent (worker): applying SMF actions for region fc9804d7-710a-4c0c-9224-dbe3d38a10e1 running snapshot 302e67d8-0457-4826-8058-830e75ae370a (state Requested)...
352025-11-18T00:07:55.241ZINFOcrucible-agent (dropshot): request completed latency_us = 22693 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:41340 req_id = 8082e529-a9e6-49dd-a5a4-61572b0c30f1 response_code = 200 uri = /crucible/0/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1/snapshots/302e67d8-0457-4826-8058-830e75ae370a/run
362025-11-18T00:07:55.248ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-fc9804d7-710a-4c0c-9224-dbe3d38a10e1-302e67d8-0457-4826-8058-830e75ae370a
372025-11-18T00:07:55.255ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-fc9804d7-710a-4c0c-9224-dbe3d38a10e1-302e67d8-0457-4826-8058-830e75ae370a
382025-11-18T00:07:55.280ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-fc9804d7-710a-4c0c-9224-dbe3d38a10e1-302e67d8-0457-4826-8058-830e75ae370a
392025-11-18T00:07:55.280ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1/.zfs/snapshot/302e67d8-0457-4826-8058-830e75ae370a
402025-11-18T00:07:55.280ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-11-18T00:07:55.280ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-11-18T00:07:55.280ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432025-11-18T00:07:55.280ZINFOcrucible-agent (worker): commit
442025-11-18T00:07:55.290ZINFOcrucible-agent (worker): ok!
452025-11-18T00:07:55.298ZINFOcrucible-agent (worker): SMF ok!
462025-11-18T00:07:55.298ZINFOcrucible-agent (datafile): region fc9804d7-710a-4c0c-9224-dbe3d38a10e1 running snapshot 302e67d8-0457-4826-8058-830e75ae370a state: Requested -> Created
472025-11-18T00:07:55.395ZINFOcrucible-agent (datafile): path is "/data/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1"
482025-11-18T00:07:55.401ZINFOcrucible-agent (datafile): dataset is oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1
492025-11-18T00:07:55.407ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1@302e67d8-0457-4826-8058-830e75ae370a
502025-11-18T00:07:55.407ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3db56757-41b9-4c06-aecc-848806347473/crucible/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1", "302e67d8-0457-4826-8058-830e75ae370a"]
512025-11-18T00:07:55.413ZINFOcrucible-agent (dropshot): request completed latency_us = 17370 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:41340 req_id = 0e07783b-f259-42bf-88be-5df260760b9e response_code = 200 uri = /crucible/0/regions/fc9804d7-710a-4c0c-9224-dbe3d38a10e1/snapshots/302e67d8-0457-4826-8058-830e75ae370a/run