{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 40997,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:05.631126767Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:51735","req_id":"6f7a0bf9-8646-4d79-8b19-118e74b1774b","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 40337,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:15.632671313Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:54087","req_id":"5c4939ad-f3f2-46af-a466-96839257a54e","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 53798,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 0,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"0.pool.ntp.org.\"),\n                ),\n                original: Query {\n                    name: Name(\"0.pool.ntp.org.\"),\n                    query_type: AAAA,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            1,\n            48,\n            4,\n            112,\n            111,\n            111,\n            108,\n            3,\n            110,\n            116,\n            112,\n            3,\n            111,\n            114,\n            103,\n            0,\n            0,\n            28,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:19.76037068Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::1b]:48749","req_id":"8b6592ce-00bb-4458-9cc0-c4db0627ed7e","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 30982,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 0,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"0.pool.ntp.org.\"),\n                ),\n                original: Query {\n                    name: Name(\"0.pool.ntp.org.\"),\n                    query_type: AAAA,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            1,\n            48,\n            4,\n            112,\n            111,\n            111,\n            108,\n            3,\n            110,\n            116,\n            112,\n            3,\n            111,\n            114,\n            103,\n            0,\n            0,\n            28,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:19.764565402Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::1b]:65522","req_id":"1739c0f7-4772-47a0-a2e3-d2f4c368f907","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 52400,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 0,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"0.pool.ntp.org.\"),\n                ),\n                original: Query {\n                    name: Name(\"0.pool.ntp.org.\"),\n                    query_type: A,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            1,\n            48,\n            4,\n            112,\n            111,\n            111,\n            108,\n            3,\n            110,\n            116,\n            112,\n            3,\n            111,\n            114,\n            103,\n            0,\n            0,\n            1,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:19.76939136Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::1b]:54028","req_id":"13bf4c5d-d846-4333-a2af-8acc6ea76926","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 42361,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:25.634455641Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:59816","req_id":"c1622c56-01ae-4253-8767-4805f744939d","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 60667,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:35.636492422Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:51117","req_id":"0bf0e677-7daa-40a1-a568-67e714b6f042","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 48069,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:45.638463602Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:61732","req_id":"5fb4f7e7-e5b5-4d8d-888c-01e732468c41","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.680056413Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:59450"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.680298675Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"211e402f-ae48-4ea2-bbaa-f70ebb1cea64","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.680321696Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"211e402f-ae48-4ea2-bbaa-f70ebb1cea64","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.680329616Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"PUT","req_id":"211e402f-ae48-4ea2-bbaa-f70ebb1cea64","remote_addr":"[fd00:1122:3344:101::b]:59450","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":178,"response_code":204}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.705796127Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:41252"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.706192031Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"919bb695-0f8c-4ff5-a96d-6cd5d4c3ce62","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.706217132Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"919bb695-0f8c-4ff5-a96d-6cd5d4c3ce62","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.706224852Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"PUT","req_id":"919bb695-0f8c-4ff5-a96d-6cd5d4c3ce62","remote_addr":"[fd00:1122:3344:101::c]:41252","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":264,"response_code":204}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.744120616Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:61094"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.744163227Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"ca623fb1-2f01-4444-be9d-3ac6cfc545e7","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.744172677Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"ca623fb1-2f01-4444-be9d-3ac6cfc545e7","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:45.744181187Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"PUT","req_id":"ca623fb1-2f01-4444-be9d-3ac6cfc545e7","remote_addr":"[fd00:1122:3344:101::a]:61094","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":233,"response_code":204}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 10796,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:46.857760316Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::c]:31956","req_id":"011bb521-dc8a-4e75-ab0d-62a3dd675bfd","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 8830,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:46.864436882Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::a]:59167","req_id":"37d87f6e-e2d3-4a4d-a39f-f645ec575938","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 31069,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:46.931639411Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::b]:16480","req_id":"bbbd8967-d3c3-4041-86e6-2c6a899c7735","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:46.996084713Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:55084"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:46.996309315Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"GET","req_id":"1de58bad-78bc-4550-949a-e75997231b60","remote_addr":"[fd00:1122:3344:101::a]:55084","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":185,"response_code":200}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:47.055171417Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:58642"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:47.05543029Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"GET","req_id":"2b46adc8-5e18-48c4-92de-5937069c0ff2","remote_addr":"[fd00:1122:3344:101::b]:58642","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":218,"response_code":200}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:47.110143751Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:37888"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:00:47.110372113Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"GET","req_id":"97d8d43a-2e68-4560-a168-967357c6c192","remote_addr":"[fd00:1122:3344:101::c]:37888","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":185,"response_code":200}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 30892,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:00:55.640373992Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:59334","req_id":"e0bbf0c1-608a-4566-812f-9b177523eb3a","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 49301,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:01:05.642450213Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:60718","req_id":"c3643e15-f2de-4dae-8e63-265026b21b05","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 63053,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:01:15.644815358Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:56612","req_id":"ad1ba35d-8055-4825-9a5d-e2bbd1b2ce04","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 2305,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:01:25.646343244Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:65386","req_id":"e5ddf69c-fbb7-423e-8303-73f193f37e52","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 20200,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:01:35.647804989Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:51831","req_id":"b7ff4027-9ee9-4485-a429-4345ad58b19b","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 34472,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:01:45.649875939Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:50179","req_id":"750ad9cd-d6e2-47e6-af11-6e0c2f9fce17","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.681140738Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:46170"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.681398671Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"e7e68235-2d8c-4127-920e-a64ce85c7ccb","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.681417241Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"e7e68235-2d8c-4127-920e-a64ce85c7ccb","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.681424081Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"PUT","req_id":"e7e68235-2d8c-4127-920e-a64ce85c7ccb","remote_addr":"[fd00:1122:3344:101::b]:46170","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":160,"response_code":204}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.707003584Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:65532"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.707346507Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"645a3e63-ae0f-4a7a-9b49-4ac324fe96f5","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.707366217Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"645a3e63-ae0f-4a7a-9b49-4ac324fe96f5","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.707379877Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"PUT","req_id":"645a3e63-ae0f-4a7a-9b49-4ac324fe96f5","remote_addr":"[fd00:1122:3344:101::c]:65532","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":222,"response_code":204}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.745012269Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:36587"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.745272332Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"014e1242-415e-401c-bfff-1bada326f5d1","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.745286632Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"new_generation":1,"req_id":"014e1242-415e-401c-bfff-1bada326f5d1","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:45.745298112Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"PUT","req_id":"014e1242-415e-401c-bfff-1bada326f5d1","remote_addr":"[fd00:1122:3344:101::a]:36587","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":235,"response_code":204}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 2586,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:01:47.031480618Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::b]:65386","req_id":"8ed77389-2bdb-47d2-9754-9d7f50fa5173","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 38292,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:01:47.031509908Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::c]:49946","req_id":"d9b10a4d-60f9-4e54-9de4-d4edd531479b","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 10752,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:01:47.112135555Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::a]:51660","req_id":"28fcfca3-4e99-4e25-835b-16b609706920","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:47.133413346Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:45766"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:47.133651188Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"GET","req_id":"8183cf1b-30fc-40b7-95db-47cc01e83ba3","remote_addr":"[fd00:1122:3344:101::b]:45766","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":182,"response_code":200}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:47.177128508Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:55350"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:47.17738619Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"GET","req_id":"ec137d9d-6b6f-4431-aa3a-251be9f373e0","remote_addr":"[fd00:1122:3344:101::c]:55350","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":215,"response_code":200}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:47.293353097Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:40012"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-19T23:01:47.293579069Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"uri":"/config","method":"GET","req_id":"624a748d-fff7-42e3-9898-f2aa83103b7b","remote_addr":"[fd00:1122:3344:101::a]:40012","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":188,"response_code":200}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 30745,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:01:55.651593008Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:62357","req_id":"deb0196e-11a9-4b1c-9242-f9cd5118bc77","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 45927,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:02:05.653058021Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:52771","req_id":"70627ffe-14ee-41d3-bd98-3995a791b45d","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 30632,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:02:15.654017753Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:56419","req_id":"4f5f4b34-0d8d-4815-8935-90accc0f8455","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 7008,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-19T23:02:25.655673408Z","hostname":"oxz_internal_dns_87a14dfa-c726-4a5b-9d9b-e173ef94dd19","pid":5486,"peer_addr":"[fd00:1122:3344:101::d]:63882","req_id":"27c9a229-4122-453e-80e5-bf4d541dcb33","component":"dns"}