{"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 40911,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"oxz_clickhouse_aec895de-ad8a-426b-ab83-7ea1d5d69229.\"),\n ),\n original: Query {\n name: Name(\"oxz_clickhouse_aec895de-ad8a-426b-ab83-7ea1d5d69229.\"),\n query_type: A,\n query_class: IN,\n },\n },\n ],\n original: [\n 51,\n 111,\n 120,\n 122,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 95,\n 97,\n 101,\n 99,\n 56,\n 57,\n 53,\n 100,\n 101,\n 45,\n 97,\n 100,\n 56,\n 97,\n 45,\n 52,\n 50,\n 54,\n 98,\n 45,\n 97,\n 98,\n 56,\n 51,\n 45,\n 55,\n 101,\n 97,\n 49,\n 100,\n 53,\n 100,\n 54,\n 57,\n 50,\n 50,\n 57,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"oxz_clickhouse_aec895de-ad8a-426b-ab83-7ea1d5d69229.\"","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:00.980781803Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::e]:53329","req_id":"50a67795-7267-4467-8187-cf217b1118f2","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 40911,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"oxz_clickhouse_aec895de-ad8a-426b-ab83-7ea1d5d69229.\"),\n ),\n original: Query {\n name: Name(\"oxz_clickhouse_aec895de-ad8a-426b-ab83-7ea1d5d69229.\"),\n query_type: A,\n query_class: IN,\n },\n },\n ],\n original: [\n 51,\n 111,\n 120,\n 122,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 95,\n 97,\n 101,\n 99,\n 56,\n 57,\n 53,\n 100,\n 101,\n 45,\n 97,\n 100,\n 56,\n 97,\n 45,\n 52,\n 50,\n 54,\n 98,\n 45,\n 97,\n 98,\n 56,\n 51,\n 45,\n 55,\n 101,\n 97,\n 49,\n 100,\n 53,\n 100,\n 54,\n 57,\n 50,\n 50,\n 57,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"oxz_clickhouse_aec895de-ad8a-426b-ab83-7ea1d5d69229.\"","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:00.981362567Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::e]:34193","req_id":"7270d5c4-a73b-45f5-96cc-305aca2d789f","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 25289,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:04.246330971Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:52404","req_id":"5cda4959-faa5-40de-ab82-727df2892408","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 37060,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:14.248040855Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:64908","req_id":"8f1cf10f-4ab5-4544-beae-a267b7ebbf16","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 46614,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:24.250075054Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:56496","req_id":"e79d5e7e-2fb2-4b2b-b941-34eb5de43a39","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:33.025037711Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:54346"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:33.025484932Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:61203"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:33.026299321Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"b76ecdde-e0e3-4fd5-ab4c-638f7de18500","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:33.026323511Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"b76ecdde-e0e3-4fd5-ab4c-638f7de18500","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:33.026338082Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"PUT","req_id":"b76ecdde-e0e3-4fd5-ab4c-638f7de18500","remote_addr":"[fd00:1122:3344:101::a]:54346","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":182,"response_code":204} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:33.026794472Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"666ca08c-a975-4d8a-b0f6-491dc9d11521","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:33.026813422Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"666ca08c-a975-4d8a-b0f6-491dc9d11521","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:33.026823293Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"PUT","req_id":"666ca08c-a975-4d8a-b0f6-491dc9d11521","remote_addr":"[fd00:1122:3344:101::b]:61203","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":240,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 8082,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:34.031580296Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::a]:13111","req_id":"500521bc-d670-4605-bafe-cc99dd8c6a50","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 20376,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:34.037835612Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::b]:20185","req_id":"28de732d-8f54-4c8e-a4f5-8184601801bd","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:34.132971532Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:60508"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:34.133210708Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"GET","req_id":"9cf32aca-51e1-4c6c-8d6f-b53184586c06","remote_addr":"[fd00:1122:3344:101::a]:60508","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":199,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:34.222924532Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:45734"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:34.223154697Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"GET","req_id":"89a8fa16-3887-44a8-8c47-c5549c43637b","remote_addr":"[fd00:1122:3344:101::b]:45734","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":199,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 54474,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:34.251344882Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:50242","req_id":"5a1c0088-3b86-42a1-8b39-b7314fdf8bdb","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:34.783718741Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:52049"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:34.784915839Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"11e0ac1f-3d2a-4c6e-b6b4-ba5305612648","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:34.78494466Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"11e0ac1f-3d2a-4c6e-b6b4-ba5305612648","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:34.78495636Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"PUT","req_id":"11e0ac1f-3d2a-4c6e-b6b4-ba5305612648","remote_addr":"[fd00:1122:3344:101::c]:52049","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":195,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 26167,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:36.40847309Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::c]:28213","req_id":"4b3290ca-fcb5-4992-939d-dbc953a96a10","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:36.50533279Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:37287"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:45:36.505586066Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"GET","req_id":"257f241a-a2c9-4b16-9df4-902939d4a5a2","remote_addr":"[fd00:1122:3344:101::c]:37287","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":213,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 19324,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:44.253223255Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:57636","req_id":"e8a42457-6b45-4bc6-b2d8-d562de01e88b","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 48991,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:45:54.255707603Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:49516","req_id":"d8158882-b4b2-4b86-8021-b3135836ca81","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 26002,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:46:04.257590965Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:62342","req_id":"961d9b67-f8e7-4c55-b55c-962749b89039","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 33265,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:46:14.259494308Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:63053","req_id":"f2e59f45-38c3-4b52-80e1-d91d93924b65","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 59854,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:46:24.26128578Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:58053","req_id":"944e7d92-de43-4124-9e63-3d6fa29a0efd","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:33.025484677Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:34707"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:33.026188403Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:58545"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:33.027063504Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"3a8feae3-dd36-40a8-b56a-8e589eb4c5b8","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:33.027081534Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"3a8feae3-dd36-40a8-b56a-8e589eb4c5b8","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:33.027089164Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"PUT","req_id":"3a8feae3-dd36-40a8-b56a-8e589eb4c5b8","remote_addr":"[fd00:1122:3344:101::a]:34707","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":174,"response_code":204} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:33.027501694Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"a3d68ddf-da36-4c1c-947c-2f7913b9d1a3","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:33.027520514Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"a3d68ddf-da36-4c1c-947c-2f7913b9d1a3","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:33.027527625Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"PUT","req_id":"a3d68ddf-da36-4c1c-947c-2f7913b9d1a3","remote_addr":"[fd00:1122:3344:101::b]:58545","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":224,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 27358,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:46:33.853948164Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::b]:15194","req_id":"7cddff13-c738-414f-bd45-23b937c477f6","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 9165,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:46:33.975468728Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::a]:53808","req_id":"4c36c424-a37d-4aef-aaae-d9aa69c3e1f6","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:33.981300583Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:38233"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:33.981508308Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"GET","req_id":"7ef88a58-ff46-4233-a1a6-e5d594ff20db","remote_addr":"[fd00:1122:3344:101::b]:38233","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":167,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:34.079206058Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:49646"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:34.079424052Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"GET","req_id":"5c0c9313-dc5e-41ef-8cf5-8292cfb4dfa8","remote_addr":"[fd00:1122:3344:101::a]:49646","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":185,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 46149,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:46:34.262554667Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:61700","req_id":"3c09df1c-8b8d-4ca3-9713-270716dfcc4d","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:34.784152705Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:55981"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:34.78523505Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"f4252be0-e0a7-4c0f-bf7b-8b31715a7d64","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:34.78525818Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"f4252be0-e0a7-4c0f-bf7b-8b31715a7d64","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:34.785265421Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"PUT","req_id":"f4252be0-e0a7-4c0f-bf7b-8b31715a7d64","remote_addr":"[fd00:1122:3344:101::c]:55981","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":185,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 5277,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:46:35.710782983Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::c]:51460","req_id":"7c8a1f29-f673-4341-aae5-00134a220f1c","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:35.889596537Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:35654"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:46:35.889832092Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"GET","req_id":"01433e93-8136-4290-9673-b8e4d8d985bf","remote_addr":"[fd00:1122:3344:101::c]:35654","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":195,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 51846,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:46:44.264262838Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:64602","req_id":"588cb8f3-a638-4502-8a63-40b0176732b0","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 1479,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:46:54.266101288Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:64598","req_id":"5224f60e-9774-49a0-a703-5c298f242e09","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 8233,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:47:04.267324766Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:49732","req_id":"a7ab48a9-8225-4ccd-9f70-a84d8ea9e946","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 23612,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:47:14.26852958Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:58822","req_id":"ddb69757-2f33-4a4d-a607-69e8f1a00f92","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 35477,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:47:24.269605237Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:52908","req_id":"2faec8cc-ba3f-496d-8215-df11ce1c4af0","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:33.025257265Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:50046"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:33.025767456Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:63845"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:33.026833761Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"f13a8b1a-03f4-42fb-9194-eb30044981d1","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:33.026847031Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"f13a8b1a-03f4-42fb-9194-eb30044981d1","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:33.026857432Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"PUT","req_id":"f13a8b1a-03f4-42fb-9194-eb30044981d1","remote_addr":"[fd00:1122:3344:101::a]:50046","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":166,"response_code":204} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:33.027046036Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"51aea874-b293-43b1-97a8-4a8c08dbd2d0","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:33.027060256Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"51aea874-b293-43b1-97a8-4a8c08dbd2d0","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:33.027071317Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"PUT","req_id":"51aea874-b293-43b1-97a8-4a8c08dbd2d0","remote_addr":"[fd00:1122:3344:101::b]:63845","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":212,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 21699,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:47:34.163168465Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::b]:31372","req_id":"487a6a47-88b3-43b9-aec3-a37a505b711a","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 6373,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:47:34.169494218Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::a]:57205","req_id":"b649e24c-a7af-4133-9199-801f4b0e350d","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:34.262897218Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:62980"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:34.263182195Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"GET","req_id":"bcb79efe-21a2-4cde-8e60-abdbba617368","remote_addr":"[fd00:1122:3344:101::b]:62980","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":240,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 14118,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:47:34.27203588Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:64485","req_id":"24d91321-acc2-43fa-9450-8605ef0ab153","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:34.304902914Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:54925"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:34.305111619Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"GET","req_id":"7592c063-0a65-449b-be6a-6e725ea9f4cd","remote_addr":"[fd00:1122:3344:101::a]:54925","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":178,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:34.783661387Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:46899"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:34.784868185Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"8482bd7e-1f39-4afc-8d44-c953a435cc82","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:34.784885535Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"new_generation":1,"req_id":"8482bd7e-1f39-4afc-8d44-c953a435cc82","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:34.784894416Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"PUT","req_id":"8482bd7e-1f39-4afc-8d44-c953a435cc82","remote_addr":"[fd00:1122:3344:101::c]:46899","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":239,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 2079,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:47:35.855596701Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::c]:55331","req_id":"b3856db8-b4c5-47bf-b51c-d10917af92ae","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:35.949372189Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:43531"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-03-10T19:47:35.949630075Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"uri":"/config","method":"GET","req_id":"ca45e333-03d4-4a56-8485-ff42194fde82","remote_addr":"[fd00:1122:3344:101::c]:43531","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":212,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 23152,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-03-10T19:47:44.274202681Z","hostname":"oxz_internal_dns_481aee8c-2536-4301-99e8-fe478ae1e659","pid":5524,"peer_addr":"[fd00:1122:3344:101::d]:58465","req_id":"85cadd12-3c60-4fe8-9716-62a6adb0b759","component":"dns"}