{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 30649,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:10.392430817Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:60135","req_id":"87b12692-2a22-4107-b065-de0c3b1f558a","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 18799,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:20.394431493Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:64018","req_id":"de726b37-5485-444f-a25e-ef8f2d876ad4","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 61102,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 0,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"0.pool.ntp.org.\"),\n                ),\n                original: Query {\n                    name: Name(\"0.pool.ntp.org.\"),\n                    query_type: AAAA,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            1,\n            48,\n            4,\n            112,\n            111,\n            111,\n            108,\n            3,\n            110,\n            116,\n            112,\n            3,\n            111,\n            114,\n            103,\n            0,\n            0,\n            28,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:27.974848943Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::1b]:33172","req_id":"c4eb422d-aa0a-495d-b819-34de23ddcc13","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 60009,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 0,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"0.pool.ntp.org.\"),\n                ),\n                original: Query {\n                    name: Name(\"0.pool.ntp.org.\"),\n                    query_type: AAAA,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            1,\n            48,\n            4,\n            112,\n            111,\n            111,\n            108,\n            3,\n            110,\n            116,\n            112,\n            3,\n            111,\n            114,\n            103,\n            0,\n            0,\n            28,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:27.978805617Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::1b]:45091","req_id":"211877ec-323a-41ca-95dc-66f8feb85094","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 12107,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 0,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"0.pool.ntp.org.\"),\n                ),\n                original: Query {\n                    name: Name(\"0.pool.ntp.org.\"),\n                    query_type: A,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            1,\n            48,\n            4,\n            112,\n            111,\n            111,\n            108,\n            3,\n            110,\n            116,\n            112,\n            3,\n            111,\n            114,\n            103,\n            0,\n            0,\n            1,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:27.983454658Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::1b]:52624","req_id":"05100681-c701-46ec-b5aa-f4a7fb211e8f","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 931,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:30.396604956Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:55356","req_id":"ce1a6d75-51d4-49f6-bebf-282ea5bf968b","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 25890,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:40.398720948Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:62148","req_id":"3fa521e5-efb1-42d5-8b22-aa6785267e66","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.115329344Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:34099"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.115660322Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"2ec77cc6-7467-4df6-a0df-258b8b3c88a4","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.115678952Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"2ec77cc6-7467-4df6-a0df-258b8b3c88a4","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.115687832Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"PUT","req_id":"2ec77cc6-7467-4df6-a0df-258b8b3c88a4","remote_addr":"[fd00:1122:3344:101::c]:34099","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":237,"response_code":204}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.11643441Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:33347"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.116798139Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"147df794-39af-45ea-9557-c81deaac9aa1","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.116817579Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"147df794-39af-45ea-9557-c81deaac9aa1","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.116828749Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"PUT","req_id":"147df794-39af-45ea-9557-c81deaac9aa1","remote_addr":"[fd00:1122:3344:101::a]:33347","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":282,"response_code":204}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.117756962Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:59200"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.118156091Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"51576ca2-cdba-4c90-b9d3-a62232268a26","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.118178442Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"51576ca2-cdba-4c90-b9d3-a62232268a26","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:48.118188252Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"PUT","req_id":"51576ca2-cdba-4c90-b9d3-a62232268a26","remote_addr":"[fd00:1122:3344:101::b]:59200","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":160,"response_code":204}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 19041,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:49.678552437Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::a]:6596","req_id":"a61acb8d-150a-4d6e-9f0b-2bc4f7d41ed6","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 21117,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:49.678581248Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::c]:32894","req_id":"4061e7dc-7724-4a10-a189-25568e9092a9","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 38144,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:49.683777742Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::b]:63850","req_id":"73d7ccc9-6e72-4e10-baea-cc03959475d5","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:49.790543173Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:55066"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:49.790757748Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"GET","req_id":"d91a0095-4fcd-4040-8149-4547983991b2","remote_addr":"[fd00:1122:3344:101::a]:55066","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":183,"response_code":200}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:49.829015009Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:64650"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:49.829216474Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"GET","req_id":"a66e0a42-7d2f-4579-afde-9f966a5b67bf","remote_addr":"[fd00:1122:3344:101::c]:64650","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":174,"response_code":200}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:49.882745288Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:46299"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:00:49.882962563Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"GET","req_id":"2f0b4888-0a9b-4cbe-8400-371528752560","remote_addr":"[fd00:1122:3344:101::b]:46299","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":185,"response_code":200}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 17582,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:00:50.400155815Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:64543","req_id":"4adc32f1-7c3f-4daa-bf2b-cc142c0d33d8","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 47454,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:01:00.401330303Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:56855","req_id":"71fbbb5e-4cb1-4051-977c-c76679a478d6","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 12609,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:01:10.403012114Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:63724","req_id":"f396e7a2-d5c3-4663-84c1-1633f0b400c9","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 33841,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:01:20.404583795Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:51524","req_id":"1ed5ade9-ec29-40ad-ac2b-561cbc2cfa40","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 13757,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:01:30.405982785Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:50659","req_id":"0330690b-50b5-422b-b46e-09c0c378251f","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 31267,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:01:40.407078992Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:58655","req_id":"e2d3aca4-9f9b-470f-918b-f19fc58e9d3f","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.11681169Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:33395"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.116996325Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"1a5747d3-f30a-4542-8c26-be86fc6908ac","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.117013315Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"1a5747d3-f30a-4542-8c26-be86fc6908ac","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.117038946Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"PUT","req_id":"1a5747d3-f30a-4542-8c26-be86fc6908ac","remote_addr":"[fd00:1122:3344:101::a]:33395","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":179,"response_code":204}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.117550118Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:64906"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.117861455Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"524b883a-f123-4a61-9008-d46f61a7eef0","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.117877976Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"524b883a-f123-4a61-9008-d46f61a7eef0","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.117886996Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"PUT","req_id":"524b883a-f123-4a61-9008-d46f61a7eef0","remote_addr":"[fd00:1122:3344:101::c]:64906","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":247,"response_code":204}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.118196593Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:46361"}
{"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.118384518Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"bb0c7061-3d20-4136-96b6-2e2d30061851","component":"store"}
{"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.118405448Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"new_generation":1,"req_id":"bb0c7061-3d20-4136-96b6-2e2d30061851","component":"store"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:48.118416898Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"PUT","req_id":"bb0c7061-3d20-4136-96b6-2e2d30061851","remote_addr":"[fd00:1122:3344:101::b]:46361","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":136,"response_code":204}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 31179,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:01:49.465030645Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::a]:38954","req_id":"9414992a-9662-46b9-93c3-9063273fd22d","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 15971,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:01:49.472036282Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::b]:63332","req_id":"1d70ff27-dc7e-4e5b-ab1f-fc872a73214c","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 59261,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            24,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            97,\n            100,\n            109,\n            105,\n            110,\n            45,\n            107,\n            101,\n            101,\n            112,\n            101,\n            114,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:01:49.52486787Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::c]:56695","req_id":"d3de2cf1-6399-4f81-a70e-8e10590000f4","component":"dns"}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:49.575237989Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:36133"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:49.575492725Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"GET","req_id":"9d72ffc7-7fe8-40b6-8149-459ebd3a6c97","remote_addr":"[fd00:1122:3344:101::a]:36133","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":203,"response_code":200}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:49.609696419Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:62874"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:49.609919415Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"GET","req_id":"b118a990-e3b5-4d45-8c9b-0f8e55f05cc0","remote_addr":"[fd00:1122:3344:101::b]:62874","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":185,"response_code":200}
{"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:49.646971557Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:64355"}
{"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-12T03:01:49.647207082Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"uri":"/config","method":"GET","req_id":"a3671a3f-81bb-4db4-ac87-a7e10d74cadf","remote_addr":"[fd00:1122:3344:101::c]:64355","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":194,"response_code":200}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 18788,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:01:50.409366995Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:54894","req_id":"a5ffc8dc-8c10-40ee-9125-95a7cdd217ce","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 36141,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:02:00.41119942Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:57217","req_id":"b1f99dc7-28dd-4ad3-be9c-0064d1d2c27b","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 32632,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:02:10.413767102Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:51545","req_id":"43f52b53-fec2-4074-a5f3-577e5a4a3e2d","component":"dns"}
{"msg":"failed to handle incoming DNS message: MessageRequest {\n    header: Header {\n        id: 59162,\n        message_type: Query,\n        op_code: Query,\n        authoritative: false,\n        truncation: false,\n        recursion_desired: true,\n        recursion_available: false,\n        authentic_data: false,\n        checking_disabled: false,\n        response_code: NoError,\n        query_count: 1,\n        answer_count: 0,\n        name_server_count: 0,\n        additional_count: 1,\n    },\n    queries: Queries {\n        queries: [\n            LowerQuery {\n                name: LowerName(\n                    Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                ),\n                original: Query {\n                    name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n                    query_type: SRV,\n                    query_class: IN,\n                },\n            },\n        ],\n        original: [\n            26,\n            95,\n            99,\n            108,\n            105,\n            99,\n            107,\n            104,\n            111,\n            117,\n            115,\n            101,\n            45,\n            99,\n            108,\n            117,\n            115,\n            116,\n            101,\n            114,\n            45,\n            110,\n            97,\n            116,\n            105,\n            118,\n            101,\n            4,\n            95,\n            116,\n            99,\n            112,\n            13,\n            99,\n            111,\n            110,\n            116,\n            114,\n            111,\n            108,\n            45,\n            112,\n            108,\n            97,\n            110,\n            101,\n            5,\n            111,\n            120,\n            105,\n            100,\n            101,\n            8,\n            105,\n            110,\n            116,\n            101,\n            114,\n            110,\n            97,\n            108,\n            0,\n            0,\n            33,\n            0,\n            1,\n        ],\n    },\n    answers: [],\n    name_servers: [],\n    additionals: [],\n    sig0: [],\n    edns: Some(\n        Edns {\n            rcode_high: 0,\n            version: 0,\n            flags: EdnsFlags {\n                dnssec_ok: false,\n                z: 0,\n            },\n            max_payload: 1232,\n            options: OPT {\n                options: [],\n            },\n        },\n    ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-12T03:02:20.416401967Z","hostname":"oxz_internal_dns_d38e7d17-75bc-4d9c-9427-34feabe8defa","pid":5516,"peer_addr":"[fd00:1122:3344:101::d]:53325","req_id":"bbcda7a9-dd81-4604-9ac2-986fb7064331","component":"dns"}