{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 47013,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:04.291811638Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::d]:63628","req_id":"4e84a55e-f252-445c-8f1d-5f5f03f330a4","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 4381,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 0,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"0.pool.ntp.org.\"),\n                ),\n                original: Query {\n                    name: Name(\"0.pool.ntp.org.\"),\n                    query_type: AAAA,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            1,\n            48,\n            4,\n            112,\n            111,\n            111,\n            108,\n            3,\n            110,\n            116,\n            112,\n            3,\n            111,\n            114,\n            103,\n            0,\n            0,\n            28,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:06.51465775Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::1b]:60780","req_id":"380ce61c-f206-4521-a0f3-b01f8d7d7e99","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 54903,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 0,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"0.pool.ntp.org.\"),\n                ),\n                original: Query {\n                    name: Name(\"0.pool.ntp.org.\"),\n                    query_type: AAAA,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            1,\n            48,\n            4,\n            112,\n            111,\n            111,\n            108,\n            3,\n            110,\n            116,\n            112,\n            3,\n            111,\n            114,\n            103,\n            0,\n            0,\n            28,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:06.518968639Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::1b]:56589","req_id":"0ff7ab59-b891-45a1-b490-4fcd770ac760","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 50916,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 0,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"0.pool.ntp.org.\"),\n                ),\n                original: Query {\n                    name: Name(\"0.pool.ntp.org.\"),\n                    query_type: A,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            1,\n            48,\n            4,\n            112,\n            111,\n            111,\n            108,\n            3,\n            110,\n            116,\n            112,\n            3,\n            111,\n            114,\n            103,\n            0,\n            0,\n            1,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:06.523288018Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::1b]:65412","req_id":"e3d650c4-85e7-4526-9a81-e5cac7c5972d","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 56450,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:14.293448533Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::d]:63920","req_id":"ea20211b-90f3-41d7-bfd3-a3925b7ccaed","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 19232,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:24.295317691Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::d]:62498","req_id":"e0dbfb10-41b1-4e06-95e6-ac7733d503ab","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 36050,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:34.296942664Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::d]:64106","req_id":"2f2d5284-186f-481e-8d36-8dbbc4c91c14","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:43.03296526Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:55879"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:43.035971258Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"new_generation":1,"req_id":"2df93635-0043-4e7b-9336-f53c15d3718d","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:43.036013898Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"new_generation":1,"req_id":"2df93635-0043-4e7b-9336-f53c15d3718d","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:43.036025068Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"uri":"/config","method":"PUT","req_id":"2df93635-0043-4e7b-9336-f53c15d3718d","remote_addr":"[fd00:1122:3344:101::c]:55879","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":267,"response_code":204}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:43.120703034Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:57235"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:43.123959736Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"new_generation":1,"req_id":"f124d469-53d3-4615-946d-1a027e0d13fd","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:43.124004017Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"new_generation":1,"req_id":"f124d469-53d3-4615-946d-1a027e0d13fd","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:43.124018697Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"uri":"/config","method":"PUT","req_id":"f124d469-53d3-4615-946d-1a027e0d13fd","remote_addr":"[fd00:1122:3344:101::b]:57235","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":330,"response_code":204}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 995,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:44.058438184Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::c]:38957","req_id":"67bcd97b-8406-49d4-9f15-f283fb89cfed","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 701,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:44.064294947Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::b]:49452","req_id":"4ef9c89b-c7db-4596-9c6d-8decc2df4262","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:44.172832845Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:42392"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:44.173092089Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"uri":"/config","method":"GET","req_id":"36545ffb-6fb9-42a6-ad0c-ef0c084e4f39","remote_addr":"[fd00:1122:3344:101::c]:42392","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":210,"response_code":200}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:44.204226978Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:42945"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:44.204464411Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"uri":"/config","method":"GET","req_id":"8dfee42f-9e12-4086-9189-67bc39e691cd","remote_addr":"[fd00:1122:3344:101::b]:42945","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":195,"response_code":200}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 36568,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:44.298849962Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::d]:61698","req_id":"9da3daa1-5e4c-44af-9439-de77e34cf134","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:44.458973235Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:58076"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:44.462226377Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"new_generation":1,"req_id":"73674984-f0bb-4fa7-8cd1-b214d73c897d","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:44.462300798Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"new_generation":1,"req_id":"73674984-f0bb-4fa7-8cd1-b214d73c897d","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:44.462328089Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"uri":"/config","method":"PUT","req_id":"73674984-f0bb-4fa7-8cd1-b214d73c897d","remote_addr":"[fd00:1122:3344:101::a]:58076","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":224,"response_code":204}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 53391,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:45.921926043Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::a]:16151","req_id":"6156d215-1e22-4e0f-8a6e-a0bb02f7422a","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:46.036789921Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:33965"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-07-06T22:30:46.037045446Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"uri":"/config","method":"GET","req_id":"d94a03a5-67b1-41b5-af5d-e7ba69b1910a","remote_addr":"[fd00:1122:3344:101::a]:33965","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":200,"response_code":200}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 11719,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-07-06T22:30:54.299791045Z","hostname":"oxz_internal_dns_39b8a3a1-a9d0-4dab-a434-9b22bb73209a","pid":5590,"peer_addr":"[fd00:1122:3344:101::d]:56412","req_id":"0b424886-c229-4ee8-995b-058720b7e4d6","component":"dns"}