{"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 51313,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"oxz_cockroachdb_c4d4f81f-1ce2-4c29-a946-a29e961a0f33.\"),\n ),\n original: Query {\n name: Name(\"oxz_cockroachdb_c4d4f81f-1ce2-4c29-a946-a29e961a0f33.\"),\n query_type: A,\n query_class: IN,\n },\n },\n ],\n original: [\n 52,\n 111,\n 120,\n 122,\n 95,\n 99,\n 111,\n 99,\n 107,\n 114,\n 111,\n 97,\n 99,\n 104,\n 100,\n 98,\n 95,\n 99,\n 52,\n 100,\n 52,\n 102,\n 56,\n 49,\n 102,\n 45,\n 49,\n 99,\n 101,\n 50,\n 45,\n 52,\n 99,\n 50,\n 57,\n 45,\n 97,\n 57,\n 52,\n 54,\n 45,\n 97,\n 50,\n 57,\n 101,\n 57,\n 54,\n 49,\n 97,\n 48,\n 102,\n 51,\n 51,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"oxz_cockroachdb_c4d4f81f-1ce2-4c29-a946-a29e961a0f33.\"","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:01.128678116Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::7]:35162","req_id":"85d2c69c-da9c-4748-8b40-8f77ac93e465","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 51313,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"oxz_cockroachdb_c4d4f81f-1ce2-4c29-a946-a29e961a0f33.\"),\n ),\n original: Query {\n name: Name(\"oxz_cockroachdb_c4d4f81f-1ce2-4c29-a946-a29e961a0f33.\"),\n query_type: A,\n query_class: IN,\n },\n },\n ],\n original: [\n 52,\n 111,\n 120,\n 122,\n 95,\n 99,\n 111,\n 99,\n 107,\n 114,\n 111,\n 97,\n 99,\n 104,\n 100,\n 98,\n 95,\n 99,\n 52,\n 100,\n 52,\n 102,\n 56,\n 49,\n 102,\n 45,\n 49,\n 99,\n 101,\n 50,\n 45,\n 52,\n 99,\n 50,\n 57,\n 45,\n 97,\n 57,\n 52,\n 54,\n 45,\n 97,\n 50,\n 57,\n 101,\n 57,\n 54,\n 49,\n 97,\n 48,\n 102,\n 51,\n 51,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"oxz_cockroachdb_c4d4f81f-1ce2-4c29-a946-a29e961a0f33.\"","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:01.129293571Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::7]:42073","req_id":"9a78bee3-5896-49cc-82f3-771333310e4c","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 56016,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:07.694186941Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:56036","req_id":"9a1eee91-ff93-40d8-8770-31ed4e093b36","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.720160863Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:63170"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.722788975Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"47d2c43c-e488-45b0-a168-c1e313508ac8","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.722816306Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"47d2c43c-e488-45b0-a168-c1e313508ac8","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.722824486Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"PUT","req_id":"47d2c43c-e488-45b0-a168-c1e313508ac8","remote_addr":"[fd00:1122:3344:101::a]:63170","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":211,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.763564408Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:65208"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.765827641Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"e9f44428-7f93-4c99-949d-71ade44fa37f","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.765864922Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"e9f44428-7f93-4c99-949d-71ade44fa37f","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.765873942Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"PUT","req_id":"e9f44428-7f93-4c99-949d-71ade44fa37f","remote_addr":"[fd00:1122:3344:101::b]:65208","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":294,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.828143122Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:52959"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.832957256Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"dc0470a5-a0f4-420e-98f4-380037dc3123","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.832996847Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"dc0470a5-a0f4-420e-98f4-380037dc3123","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:16.833011277Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"PUT","req_id":"dc0470a5-a0f4-420e-98f4-380037dc3123","remote_addr":"[fd00:1122:3344:101::c]:52959","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":373,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 39428,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:17.696610004Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:54892","req_id":"46522aa8-7d44-4059-b946-73b89a5168d8","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 4473,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:18.111040558Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::c]:31395","req_id":"c22e89a9-0b77-410a-a476-619492965424","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 44856,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:18.113870535Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::b]:54769","req_id":"77038609-ebf9-4f36-b9e7-cf22f192c0b7","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 51169,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:18.167266316Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::a]:25626","req_id":"37bd8483-6204-4d02-9f77-692df1964ef5","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:18.210951687Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:50890"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:18.211227294Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"GET","req_id":"148a5abf-6ad2-4ee2-9360-0ee47f45f69a","remote_addr":"[fd00:1122:3344:101::b]:50890","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":212,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:18.26104616Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:64355"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:18.261320866Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"GET","req_id":"970b3567-5697-4f6c-9527-43730ddb21f5","remote_addr":"[fd00:1122:3344:101::a]:64355","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":227,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:18.301861163Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:59875"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:45:18.302089199Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"GET","req_id":"8bdbae7a-821e-4c20-9978-0b19a4b72dc3","remote_addr":"[fd00:1122:3344:101::c]:59875","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":184,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 1155,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:27.6980608Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:64974","req_id":"7143ffb2-b45b-4f4b-8743-28b1f189c7ad","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 49284,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:37.699071647Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:57205","req_id":"db7f5f56-2679-4b22-95ed-a6199043bbae","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 13901,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:47.70096661Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:60743","req_id":"12252e96-3c77-4cdc-ad18-3e048026d35e","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 56205,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:45:57.701956672Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:52263","req_id":"fcf9e433-2ab4-4388-8867-f06a7dee8790","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 40674,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:46:07.703061527Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:53134","req_id":"5cb0e59c-18d0-47a7-9fae-58a1f82578d1","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.719505065Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:43499"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.721866901Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"cb981ee5-d1cd-45e8-b8a7-24c4af29bf26","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.721891371Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"cb981ee5-d1cd-45e8-b8a7-24c4af29bf26","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.721899112Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"PUT","req_id":"cb981ee5-d1cd-45e8-b8a7-24c4af29bf26","remote_addr":"[fd00:1122:3344:101::a]:43499","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":256,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.766704529Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:40473"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.770043238Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"b0c03fc1-ce43-4e21-8881-ef3b7cb3b6bf","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.770080029Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"b0c03fc1-ce43-4e21-8881-ef3b7cb3b6bf","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.77009184Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"PUT","req_id":"b0c03fc1-ce43-4e21-8881-ef3b7cb3b6bf","remote_addr":"[fd00:1122:3344:101::b]:40473","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":209,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.830660879Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:38388"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.83367199Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"ae505b09-2bfc-404d-8e8f-a17050d907c4","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.833717811Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"ae505b09-2bfc-404d-8e8f-a17050d907c4","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:16.833736652Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"PUT","req_id":"ae505b09-2bfc-404d-8e8f-a17050d907c4","remote_addr":"[fd00:1122:3344:101::c]:38388","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":256,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 2995,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:46:17.704246813Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:50962","req_id":"5ed994c3-1e72-4ff2-874a-beb90d65fb06","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 64265,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:46:17.921228305Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::b]:34067","req_id":"81ca6d3b-8ccf-42be-a40b-4ae029d70682","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 28000,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:46:17.92144606Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::a]:42728","req_id":"880b27b6-da6c-4a43-a4d4-f1f3644e1660","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 23187,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:46:17.927863142Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::c]:59624","req_id":"9c1712cb-6a37-4960-8bef-aa496cad19a0","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:18.032366779Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:34875"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:18.032612145Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"GET","req_id":"3a3c0c3a-f60d-4e53-882b-06d981ab87b2","remote_addr":"[fd00:1122:3344:101::a]:34875","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":199,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:18.068349478Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:33091"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:18.068601474Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"GET","req_id":"acad1664-2b5a-4d86-ba1c-aea1e49107ee","remote_addr":"[fd00:1122:3344:101::c]:33091","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":214,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:18.116803592Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:49882"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:46:18.117038368Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"GET","req_id":"23ea0997-ac6b-4457-9595-73eccbddcc87","remote_addr":"[fd00:1122:3344:101::b]:49882","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":193,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 53338,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:46:27.705931622Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:56187","req_id":"11c974d6-1f7f-4bf2-b331-6a849d162f43","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 51121,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:46:37.70763725Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:51337","req_id":"d22d5ef6-c665-4b33-925e-c1d07ce96c7a","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 6496,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:46:47.709420434Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:63551","req_id":"a23f0a6b-dc21-4e5d-9890-4b9f5a321068","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 56080,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:46:57.710339891Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:55460","req_id":"5df64b4f-a001-46d9-8c07-9e4baa0bf16e","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 39360,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:07.711407895Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:58172","req_id":"be381763-eeae-446f-b47b-499b87c72ddd","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.720057841Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:58206"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.722390336Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"ade8bb48-2716-45aa-b1f1-92b01dd58039","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.722412317Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"ade8bb48-2716-45aa-b1f1-92b01dd58039","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.722419857Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"PUT","req_id":"ade8bb48-2716-45aa-b1f1-92b01dd58039","remote_addr":"[fd00:1122:3344:101::a]:58206","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":232,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.764813308Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:63182"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.768014694Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"0a5d5fc9-46cf-4df4-ba53-75a0658b8ba8","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.768052965Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"0a5d5fc9-46cf-4df4-ba53-75a0658b8ba8","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.768065965Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"PUT","req_id":"0a5d5fc9-46cf-4df4-ba53-75a0658b8ba8","remote_addr":"[fd00:1122:3344:101::b]:63182","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":286,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.827780955Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:59979"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.831206765Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"c51a1975-531e-44fa-8aee-08a38d4e1196","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.831248486Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"new_generation":1,"req_id":"c51a1975-531e-44fa-8aee-08a38d4e1196","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:16.831260647Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"PUT","req_id":"c51a1975-531e-44fa-8aee-08a38d4e1196","remote_addr":"[fd00:1122:3344:101::c]:59979","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":232,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 15653,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:17.712620384Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:57851","req_id":"6f811253-3c96-4f45-bc34-cc2763992fe6","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 29044,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:18.096073587Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::a]:34768","req_id":"7d6c9ed2-9368-4837-9b4e-3181a65b253c","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 31430,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:18.101968166Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::c]:10967","req_id":"9279f908-2b4c-40fa-a76d-a1af00453fb0","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 61839,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:18.173312901Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::b]:62416","req_id":"89b7c7f8-6534-4970-8150-ff9ea3a78d03","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:18.202351456Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:36352"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:18.202794207Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"GET","req_id":"a7b280a0-14b7-4945-84f0-93cc8183e717","remote_addr":"[fd00:1122:3344:101::a]:36352","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":232,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:18.245134976Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:59782"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:18.245426553Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"GET","req_id":"56d96b76-e85e-4007-bf38-0a7e82183d93","remote_addr":"[fd00:1122:3344:101::c]:59782","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":245,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:18.29232126Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:50224"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-03T01:47:18.292564546Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"uri":"/config","method":"GET","req_id":"9ffc0aec-3acf-410d-a3c2-37dc6c4ad18e","remote_addr":"[fd00:1122:3344:101::b]:50224","local_addr":"[fd00:1122:3344:1::1]:5353","component":"http","latency_us":204,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 24420,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:27.714656382Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:51760","req_id":"2d8affc6-0fd4-4662-8c98-3d9ed6edfc11","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 13202,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:37.716973046Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:50309","req_id":"e8875a5f-4bc3-4d86-833f-9dc49be49d6b","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 5262,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:47.71890417Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::d]:60710","req_id":"6653072f-c084-4ca3-adc7-82988fb0701a","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 32788,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"0.pool.ntp.org.\"),\n ),\n original: Query {\n name: Name(\"0.pool.ntp.org.\"),\n query_type: AAAA,\n query_class: IN,\n },\n },\n ],\n original: [\n 1,\n 48,\n 4,\n 112,\n 111,\n 111,\n 108,\n 3,\n 110,\n 116,\n 112,\n 3,\n 111,\n 114,\n 103,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:51.758789533Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::1b]:45462","req_id":"d87eb41e-aa0a-4881-9c20-5deceacc034d","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 47281,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"0.pool.ntp.org.\"),\n ),\n original: Query {\n name: Name(\"0.pool.ntp.org.\"),\n query_type: AAAA,\n query_class: IN,\n },\n },\n ],\n original: [\n 1,\n 48,\n 4,\n 112,\n 111,\n 111,\n 108,\n 3,\n 110,\n 116,\n 112,\n 3,\n 111,\n 114,\n 103,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:51.762659914Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::1b]:59102","req_id":"57d545f8-2b61-414d-a430-be1ecc124d6d","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 60443,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"0.pool.ntp.org.\"),\n ),\n original: Query {\n name: Name(\"0.pool.ntp.org.\"),\n query_type: A,\n query_class: IN,\n },\n },\n ],\n original: [\n 1,\n 48,\n 4,\n 112,\n 111,\n 111,\n 108,\n 3,\n 110,\n 116,\n 112,\n 3,\n 111,\n 114,\n 103,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-03-03T01:47:51.766779532Z","hostname":"oxz_internal_dns_096b7a76-9541-4232-b4c8-47ac47f0735f","pid":5479,"peer_addr":"[fd00:1122:3344:101::1b]:60239","req_id":"ba756ca5-b751-4cf8-93ad-1017b650cbaa","component":"dns"}