{"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 59575,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:02.103807886Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:57817","req_id":"313ea39b-71c5-4853-99f7-948a2ab99bda","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 41325,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:12.104990969Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:61260","req_id":"e1b96c78-b744-4228-b774-31064e405766","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:13.064340158Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:62294"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:13.064619045Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"0521fdcc-54d0-4e56-b7ec-c224d45ff660","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:13.064647425Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"0521fdcc-54d0-4e56-b7ec-c224d45ff660","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:13.064660535Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"PUT","req_id":"0521fdcc-54d0-4e56-b7ec-c224d45ff660","remote_addr":"[fd00:1122:3344:101::a]:62294","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":206,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 10876,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:13.312195912Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::a]:56649","req_id":"b92d1650-fc3e-4fcb-a273-05ef2b42da8b","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:13.42298874Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:54284"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:13.423208075Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"3da60b31-b6d9-4537-a39d-06bd15e65ace","remote_addr":"[fd00:1122:3344:101::a]:54284","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":189,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.014966027Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:64781"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.015242724Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"e3794f0c-c668-43a1-8113-f91aec839394","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.015272574Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"e3794f0c-c668-43a1-8113-f91aec839394","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.015281875Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"PUT","req_id":"e3794f0c-c668-43a1-8113-f91aec839394","remote_addr":"[fd00:1122:3344:101::c]:64781","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":230,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 29520,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:14.03426536Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::a]:24826","req_id":"4297866b-9e8d-47a1-aba1-d1a6539dcf28","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.049514516Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:40453"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.050005078Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"32bdc3e2-c801-4b62-aa4c-2a5f98e1e05d","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.050041768Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"32bdc3e2-c801-4b62-aa4c-2a5f98e1e05d","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.050052488Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"PUT","req_id":"32bdc3e2-c801-4b62-aa4c-2a5f98e1e05d","remote_addr":"[fd00:1122:3344:101::b]:40453","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":355,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 21265,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:14.101434721Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::b]:57417","req_id":"d067dcba-508a-4ef5-a3fe-113ba2b713cb","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 18135,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:14.154083774Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::c]:58759","req_id":"de3f167b-1b1d-4dd4-9d13-a85ff9e22c10","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.22438898Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:44309"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.225079006Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"f357521d-e583-47e6-a92c-c4f3003e9b0c","remote_addr":"[fd00:1122:3344:101::a]:44309","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":294,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.38202618Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:50611"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.38244538Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"c87a24a5-ea9e-4c05-81a4-8ed91e2942d5","remote_addr":"[fd00:1122:3344:101::b]:50611","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":302,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.44750088Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:49616"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:14.447807268Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"919009fc-fe5b-4315-82f0-95645487e4c6","remote_addr":"[fd00:1122:3344:101::c]:49616","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":262,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 19777,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:15.298876649Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::c]:1129","req_id":"35557678-e635-427d-9ec1-48079cce75ca","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 58323,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:15.299241427Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::b]:2323","req_id":"cf37fdf5-f002-4d73-9b6d-e2cd669dd4db","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:15.411955211Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:36942"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:15.412218987Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"356319d7-eb7f-454f-bff3-9f52025f9860","remote_addr":"[fd00:1122:3344:101::b]:36942","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":216,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:15.448465066Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:48705"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:00:15.448729843Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"9254ecbe-08ec-42f0-ae6a-f6f845983d88","remote_addr":"[fd00:1122:3344:101::c]:48705","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":205,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 20903,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:22.106343084Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:50818","req_id":"39c10a2c-2754-4611-91ca-3d90139414a5","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 47764,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:32.108028125Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:55489","req_id":"dfc850ce-d1f9-4ecb-a0c0-7060a56b507e","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 27971,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:42.110383051Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:51890","req_id":"9bb2b03e-baba-4093-9308-c87a4b13c20b","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 9257,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:00:52.112157755Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:56014","req_id":"81d94a04-bbb2-4972-9c88-4f043713f1b1","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 33816,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:01:02.113488869Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:50500","req_id":"d90a8d95-78ce-43f7-b510-0938d3fbb1d7","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 30879,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:01:12.114975596Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:60277","req_id":"ddfef0f6-fdaa-41e0-9b78-b0e8f1e0f32c","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:13.063605367Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:35199"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:13.063897464Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"a3d1cd65-ff82-4a4f-bc02-2e0a406de292","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:13.063916434Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"a3d1cd65-ff82-4a4f-bc02-2e0a406de292","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:13.063926984Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"PUT","req_id":"a3d1cd65-ff82-4a4f-bc02-2e0a406de292","remote_addr":"[fd00:1122:3344:101::a]:35199","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":229,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 30786,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:01:13.878780337Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::a]:52997","req_id":"988ff391-8801-4a62-9024-864376ae2e1f","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:13.985149138Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:49665"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:13.985411554Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"96ea1f6a-d8dd-47e2-bf21-faff9a293083","remote_addr":"[fd00:1122:3344:101::a]:49665","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":217,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:14.015136447Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:33416"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:14.015403043Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"bdb65761-b4f8-4638-b03d-cd14966407cd","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:14.015442244Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"bdb65761-b4f8-4638-b03d-cd14966407cd","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:14.015452664Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"PUT","req_id":"bdb65761-b4f8-4638-b03d-cd14966407cd","remote_addr":"[fd00:1122:3344:101::c]:33416","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":254,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:14.043071697Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:64141"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:14.043516628Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"b7a9626a-74e0-4e35-86b7-bc94b1dea7e2","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:14.043552978Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"b7a9626a-74e0-4e35-86b7-bc94b1dea7e2","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:14.043566059Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"PUT","req_id":"b7a9626a-74e0-4e35-86b7-bc94b1dea7e2","remote_addr":"[fd00:1122:3344:101::b]:64141","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":267,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 1912,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:01:14.874254841Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::c]:40065","req_id":"1e3d4e4f-c4ef-4cab-9d7a-d3ff9a5ce062","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:14.975447028Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:53375"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:14.975771086Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"fb9ca3ee-2131-4e31-8b5f-a1f260f3148c","remote_addr":"[fd00:1122:3344:101::c]:53375","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":216,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 44063,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:01:15.366479006Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::b]:61691","req_id":"82d3f460-e1b7-4644-8252-636a3f081e6d","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:15.548069771Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:40111"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:01:15.548315287Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"fa939acb-2b9b-4905-983e-4dd6931eec3b","remote_addr":"[fd00:1122:3344:101::b]:40111","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":212,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 36977,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:01:22.117544805Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:64670","req_id":"aec8c040-de06-4226-8d14-b41ca5e46169","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 43869,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:01:32.118313518Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:60951","req_id":"13e745e9-997e-404c-9699-555c3fc92981","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 18410,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:01:42.120160811Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:61485","req_id":"667d78b2-9eb5-4854-853d-82a9646de0d9","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 65449,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:01:52.121835114Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:52432","req_id":"160bdfc0-5266-40dc-85de-e2e193a79a5c","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 34596,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:02:02.124315593Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:60848","req_id":"d4b93830-cb44-45ec-819e-e3e587a6cd6f","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 32050,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:02:12.125736119Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:59493","req_id":"73a08fd3-4ff5-4079-8940-7af6e1f96477","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:13.06268513Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:58595"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:13.062972307Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"e7cef9b3-c1bb-4d22-82da-359e45535562","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:13.063007728Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"e7cef9b3-c1bb-4d22-82da-359e45535562","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:13.063017228Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"PUT","req_id":"e7cef9b3-c1bb-4d22-82da-359e45535562","remote_addr":"[fd00:1122:3344:101::a]:58595","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":200,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 36041,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:02:13.826760485Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::a]:23706","req_id":"b0a2312d-ed9a-45cc-8cfc-b5c30db795f8","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:13.926442646Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:52718"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:13.926674132Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"213abfed-1cfc-498d-9ecc-3115daac3552","remote_addr":"[fd00:1122:3344:101::a]:52718","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":200,"response_code":200} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:14.014346355Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:34094"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:14.014669712Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"24aa65df-d654-40c6-8368-b8a88c85ca75","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:14.014696873Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"24aa65df-d654-40c6-8368-b8a88c85ca75","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:14.014707713Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"PUT","req_id":"24aa65df-d654-40c6-8368-b8a88c85ca75","remote_addr":"[fd00:1122:3344:101::c]:34094","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":295,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:14.042007508Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:56783"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:14.042428608Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"de4aabe6-8a16-4e8c-b4d1-db607134f3eb","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:14.042456229Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"new_generation":1,"req_id":"de4aabe6-8a16-4e8c-b4d1-db607134f3eb","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:14.042465959Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"PUT","req_id":"de4aabe6-8a16-4e8c-b4d1-db607134f3eb","remote_addr":"[fd00:1122:3344:101::b]:56783","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":205,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 11789,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:02:14.740300425Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::c]:16734","req_id":"ee48d2df-42ae-4941-9117-8a3ceee2499b","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:14.840903538Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:36628"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:14.841131323Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"f4f61d8d-8fc0-4c82-91dc-7e10214a6b29","remote_addr":"[fd00:1122:3344:101::c]:36628","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":195,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 25119,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-admin-keeper._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:02:15.168909645Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::b]:16367","req_id":"82f018a3-c561-47fb-b212-47407b331c26","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:15.270613374Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:51249"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2026-06-18T23:02:15.27086814Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"uri":"/config","method":"GET","req_id":"97ba85bb-a1fa-4cc8-a79f-5ec084270d11","remote_addr":"[fd00:1122:3344:101::b]:51249","local_addr":"[fd00:1122:3344:2::1]:5353","component":"http","latency_us":211,"response_code":200} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 60403,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:02:22.128787193Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:63349","req_id":"2bc54bdf-ce4a-42a8-afc2-113d2fa0aa16","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 63974,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"0.pool.ntp.org.\"),\n ),\n original: Query {\n name: Name(\"0.pool.ntp.org.\"),\n query_type: AAAA,\n query_class: IN,\n },\n },\n ],\n original: [\n 1,\n 48,\n 4,\n 112,\n 111,\n 111,\n 108,\n 3,\n 110,\n 116,\n 112,\n 3,\n 111,\n 114,\n 103,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:02:24.253998541Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::1b]:45270","req_id":"499fc151-e828-4085-90fc-977bae5b10fd","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 5364,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"0.pool.ntp.org.\"),\n ),\n original: Query {\n name: Name(\"0.pool.ntp.org.\"),\n query_type: AAAA,\n query_class: IN,\n },\n },\n ],\n original: [\n 1,\n 48,\n 4,\n 112,\n 111,\n 111,\n 108,\n 3,\n 110,\n 116,\n 112,\n 3,\n 111,\n 114,\n 103,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:02:24.259388531Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::1b]:51024","req_id":"df4f1db0-ba35-409c-a6ff-d42cd4969b87","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 13931,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"0.pool.ntp.org.\"),\n ),\n original: Query {\n name: Name(\"0.pool.ntp.org.\"),\n query_type: A,\n query_class: IN,\n },\n },\n ],\n original: [\n 1,\n 48,\n 4,\n 112,\n 111,\n 111,\n 108,\n 3,\n 110,\n 116,\n 112,\n 3,\n 111,\n 114,\n 103,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"0.pool.ntp.org.\"","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:02:24.263842357Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::1b]:45943","req_id":"1a21b776-dc5b-43f3-9df2-debf5c60897c","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 17828,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n queries: Queries {\n queries: [\n LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n ],\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n flags: EdnsFlags {\n dnssec_ok: false,\n z: 0,\n },\n max_payload: 1232,\n options: OPT {\n options: [],\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2026-06-18T23:02:32.131322975Z","hostname":"oxz_internal_dns_a2a795ca-5848-4250-9b79-77ae27e76e3b","pid":5407,"peer_addr":"[fd00:1122:3344:101::d]:55572","req_id":"2a0f428c-ebe3-4299-ae37-0bfd12686f28","component":"dns"}