{"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 48379,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"oxz_clickhouse_31a039b7-ad1f-484f-89fb-bb5113fecb05.\"),\n ),\n original: Query {\n name: Name(\"oxz_clickhouse_31a039b7-ad1f-484f-89fb-bb5113fecb05.\"),\n query_type: A,\n query_class: IN,\n },\n },\n ],\n original: [\n 51,\n 111,\n 120,\n 122,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 95,\n 51,\n 49,\n 97,\n 48,\n 51,\n 57,\n 98,\n 55,\n 45,\n 97,\n 100,\n 49,\n 102,\n 45,\n 52,\n 56,\n 52,\n 102,\n 45,\n 56,\n 57,\n 102,\n 98,\n 45,\n 98,\n 98,\n 53,\n 49,\n 49,\n 51,\n 102,\n 101,\n 99,\n 98,\n 48,\n 53,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"oxz_clickhouse_31a039b7-ad1f-484f-89fb-bb5113fecb05.\"","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:01.013246034Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::e]:44272","req_id":"e20787b6-420a-47a9-b621-9edc1854d0d7","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 48379,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"oxz_clickhouse_31a039b7-ad1f-484f-89fb-bb5113fecb05.\"),\n ),\n original: Query {\n name: Name(\"oxz_clickhouse_31a039b7-ad1f-484f-89fb-bb5113fecb05.\"),\n query_type: A,\n query_class: IN,\n },\n },\n ],\n original: [\n 51,\n 111,\n 120,\n 122,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 95,\n 51,\n 49,\n 97,\n 48,\n 51,\n 57,\n 98,\n 55,\n 45,\n 97,\n 100,\n 49,\n 102,\n 45,\n 52,\n 56,\n 52,\n 102,\n 45,\n 56,\n 57,\n 102,\n 98,\n 45,\n 98,\n 98,\n 53,\n 49,\n 49,\n 51,\n 102,\n 101,\n 99,\n 98,\n 48,\n 53,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"oxz_clickhouse_31a039b7-ad1f-484f-89fb-bb5113fecb05.\"","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:01.013932529Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::e]:35897","req_id":"c171b405-44dd-47c8-b7ec-6663e064d76a","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 8722,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:03.915117145Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:60156","req_id":"0e37df2a-a970-4d23-8685-73c3a2a81491","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 24886,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:13.916701917Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:58616","req_id":"6673da2d-9230-4578-a2cb-d699db18ad0a","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 52672,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:23.918884242Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:57575","req_id":"cc9ca849-051f-4b37-a738-bb7b836b21ce","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.088384647Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:54384"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.088778796Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"133e78b9-1efe-476d-9861-f6f5584345c0","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.088798906Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"133e78b9-1efe-476d-9861-f6f5584345c0","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.088809857Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"PUT","req_id":"133e78b9-1efe-476d-9861-f6f5584345c0","remote_addr":"[fd00:1122:3344:101::c]:54384","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":297,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.094851482Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:41337"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.095117628Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"1195cffa-2538-42a1-bb0a-8a49409908d2","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.095134399Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"1195cffa-2538-42a1-bb0a-8a49409908d2","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.095145619Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"PUT","req_id":"1195cffa-2538-42a1-bb0a-8a49409908d2","remote_addr":"[fd00:1122:3344:101::a]:41337","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":160,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.118217898Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:65155"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.118544835Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"1da777bd-2bb7-40c5-9dea-b0d549568108","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.118574876Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"1da777bd-2bb7-40c5-9dea-b0d549568108","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:33.118586876Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"PUT","req_id":"1da777bd-2bb7-40c5-9dea-b0d549568108","remote_addr":"[fd00:1122:3344:101::b]:65155","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":246,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 7754,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:33.920233869Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:55005","req_id":"62c865a6-bf94-4751-b059-1c136313f89f","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 41674,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:34.643227443Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::c]:27130","req_id":"8b38c226-7e66-48ff-9eff-a707efc8a4dd","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 54925,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:34.649033964Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::b]:33598","req_id":"598a4e4d-047a-495c-a4f4-3a921f39c35e","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 58470,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:34.699957719Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::a]:47053","req_id":"a720f500-4805-4029-98c3-c6c2cffc7775","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:34.755827425Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:39509"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:34.75605062Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"GET","req_id":"dd23ba35-4ae5-4b37-91ea-5c023eeb9d94","remote_addr":"[fd00:1122:3344:101::b]:39509","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":189,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:34.840746984Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:62695"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:34.840974079Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"GET","req_id":"2d95f2fa-47f3-459d-803b-989428e140c7","remote_addr":"[fd00:1122:3344:101::c]:62695","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":193,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:34.898714537Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:59766"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:45:34.898949583Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"GET","req_id":"e982d256-22b4-4090-836a-69804ed31380","remote_addr":"[fd00:1122:3344:101::a]:59766","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":199,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 3450,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:43.922034816Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:52044","req_id":"a3112171-e68e-49b0-866f-6bd9badd4ad1","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 22040,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:53.923368293Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:63762","req_id":"668c63df-a2d9-4f65-aea9-70d3f0b0140a","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 25254,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"0.pool.ntp.org.\"),\n ),\n original: Query {\n name: Name(\"0.pool.ntp.org.\"),\n query_type: AAAA,\n query_class: IN,\n },\n },\n ],\n original: [\n 1,\n 48,\n 4,\n 112,\n 111,\n 111,\n 108,\n 3,\n 110,\n 116,\n 112,\n 3,\n 111,\n 114,\n 103,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:56.171868666Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::1b]:33733","req_id":"8885d81e-5b3a-45c7-ae48-0d4961b23eb4","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 17997,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"0.pool.ntp.org.\"),\n ),\n original: Query {\n name: Name(\"0.pool.ntp.org.\"),\n query_type: AAAA,\n query_class: IN,\n },\n },\n ],\n original: [\n 1,\n 48,\n 4,\n 112,\n 111,\n 111,\n 108,\n 3,\n 110,\n 116,\n 112,\n 3,\n 111,\n 114,\n 103,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:56.176039689Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::1b]:53835","req_id":"ed686edc-9e5a-4cd8-89e9-ba2b9ed62b1c","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 38640,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"0.pool.ntp.org.\"),\n ),\n original: Query {\n name: Name(\"0.pool.ntp.org.\"),\n query_type: A,\n query_class: IN,\n },\n },\n ],\n original: [\n 1,\n 48,\n 4,\n 112,\n 111,\n 111,\n 108,\n 3,\n 110,\n 116,\n 112,\n 3,\n 111,\n 114,\n 103,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:45:56.18007373Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::1b]:38204","req_id":"973329d4-c76a-4747-9cca-baa44610e9f7","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 58627,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:46:03.925155859Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:63247","req_id":"6bf32579-30ca-4eb1-9b8d-90980a5f3fa0","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 34381,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:46:13.927139899Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:62978","req_id":"7649a2e8-3a62-4cbd-bd81-b22054401764","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 61757,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:46:23.928326192Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:60157","req_id":"62230f64-6e19-4011-a367-53d166a3fdb7","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.088008785Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:34812"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.088278981Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"7acece30-cc28-4d52-82a5-49c3381e374b","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.088298542Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"7acece30-cc28-4d52-82a5-49c3381e374b","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.088310512Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"PUT","req_id":"7acece30-cc28-4d52-82a5-49c3381e374b","remote_addr":"[fd00:1122:3344:101::c]:34812","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":165,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.089867037Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:33887"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.090223285Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"ceb4a9b2-3c21-4a14-9be1-410c2fd78d77","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.090238765Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"ceb4a9b2-3c21-4a14-9be1-410c2fd78d77","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.090248895Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"PUT","req_id":"ceb4a9b2-3c21-4a14-9be1-410c2fd78d77","remote_addr":"[fd00:1122:3344:101::a]:33887","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":209,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.118181804Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:58791"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.118727586Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"b13469d9-b5b5-478f-a40c-706f490264d6","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.118763787Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"new_generation":1,"req_id":"b13469d9-b5b5-478f-a40c-706f490264d6","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:33.118774197Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"PUT","req_id":"b13469d9-b5b5-478f-a40c-706f490264d6","remote_addr":"[fd00:1122:3344:101::b]:58791","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":409,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 6515,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:46:33.930147239Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:56286","req_id":"9ea77ce3-86f3-4118-bc98-82abc076ab93","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 11578,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:46:37.200545146Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::a]:15757","req_id":"695b569e-f815-4b7a-9042-e55fdff213cb","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 65256,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:46:37.206588422Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::c]:35561","req_id":"a0aaa731-b17e-4d37-8217-8d3cfb7400d8","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 30135,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:46:37.206785526Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::b]:16989","req_id":"bc74b7b7-2753-44a8-b3c3-c9b5584ba371","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:37.410893434Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:64815"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:37.411108749Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"GET","req_id":"acaa6bb5-288a-4e80-8f7e-2f30f239e167","remote_addr":"[fd00:1122:3344:101::a]:64815","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":187,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:37.454967935Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:58537"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:37.45517397Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"GET","req_id":"cd94760b-bf6b-43e0-a23f-31a8aa786487","remote_addr":"[fd00:1122:3344:101::b]:58537","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":179,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:37.490177177Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:42579"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-05-02T04:46:37.490442373Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"uri":"/config","method":"GET","req_id":"add51c90-1e2a-4ed6-808f-d7bcdc8f19b9","remote_addr":"[fd00:1122:3344:101::c]:42579","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":210,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 162,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:46:43.93216144Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:60174","req_id":"d129bd25-7159-4516-be5b-deeac7165142","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 49534,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-05-02T04:46:53.933917725Z","hostname":"oxz_internal_dns_b70cd165-e347-4c88-a6e2-a850ae434e10","pid":5545,"peer_addr":"[fd00:1122:3344:101::d]:62631","req_id":"7162bc7a-f6d6-4fb8-b6c7-87abef416250","component":"dns"}