{"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 1203,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"oxz_cockroachdb_a3a3be2e-fb0d-41d0-9bb7-24ce857f9eb3.\"),\n ),\n original: Query {\n name: Name(\"oxz_cockroachdb_a3a3be2e-fb0d-41d0-9bb7-24ce857f9eb3.\"),\n query_type: A,\n query_class: IN,\n },\n },\n original: [\n 52,\n 111,\n 120,\n 122,\n 95,\n 99,\n 111,\n 99,\n 107,\n 114,\n 111,\n 97,\n 99,\n 104,\n 100,\n 98,\n 95,\n 97,\n 51,\n 97,\n 51,\n 98,\n 101,\n 50,\n 101,\n 45,\n 102,\n 98,\n 48,\n 100,\n 45,\n 52,\n 49,\n 100,\n 48,\n 45,\n 57,\n 98,\n 98,\n 55,\n 45,\n 50,\n 52,\n 99,\n 101,\n 56,\n 53,\n 55,\n 102,\n 57,\n 101,\n 98,\n 51,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"oxz_cockroachdb_a3a3be2e-fb0d-41d0-9bb7-24ce857f9eb3.\"","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:30:01.007901198Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::7]:59423","req_id":"baeb63a9-6cfe-4a91-8e65-3680aaeda74b","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 1203,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"oxz_cockroachdb_a3a3be2e-fb0d-41d0-9bb7-24ce857f9eb3.\"),\n ),\n original: Query {\n name: Name(\"oxz_cockroachdb_a3a3be2e-fb0d-41d0-9bb7-24ce857f9eb3.\"),\n query_type: A,\n query_class: IN,\n },\n },\n original: [\n 52,\n 111,\n 120,\n 122,\n 95,\n 99,\n 111,\n 99,\n 107,\n 114,\n 111,\n 97,\n 99,\n 104,\n 100,\n 98,\n 95,\n 97,\n 51,\n 97,\n 51,\n 98,\n 101,\n 50,\n 101,\n 45,\n 102,\n 98,\n 48,\n 100,\n 45,\n 52,\n 49,\n 100,\n 48,\n 45,\n 57,\n 98,\n 98,\n 55,\n 45,\n 50,\n 52,\n 99,\n 101,\n 56,\n 53,\n 55,\n 102,\n 57,\n 101,\n 98,\n 51,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"oxz_cockroachdb_a3a3be2e-fb0d-41d0-9bb7-24ce857f9eb3.\"","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:30:01.008130985Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::7]:59913","req_id":"240345c2-8e4f-4db7-8f25-f2a3ef191dc6","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 23187,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"oxz_cockroachdb_54b02a38-ac6b-4866-bfe6-cc383b36872d.\"),\n ),\n original: Query {\n name: Name(\"oxz_cockroachdb_54b02a38-ac6b-4866-bfe6-cc383b36872d.\"),\n query_type: A,\n query_class: IN,\n },\n },\n original: [\n 52,\n 111,\n 120,\n 122,\n 95,\n 99,\n 111,\n 99,\n 107,\n 114,\n 111,\n 97,\n 99,\n 104,\n 100,\n 98,\n 95,\n 53,\n 52,\n 98,\n 48,\n 50,\n 97,\n 51,\n 56,\n 45,\n 97,\n 99,\n 54,\n 98,\n 45,\n 52,\n 56,\n 54,\n 54,\n 45,\n 98,\n 102,\n 101,\n 54,\n 45,\n 99,\n 99,\n 51,\n 56,\n 51,\n 98,\n 51,\n 54,\n 56,\n 55,\n 50,\n 100,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"oxz_cockroachdb_54b02a38-ac6b-4866-bfe6-cc383b36872d.\"","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:30:01.027110598Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::6]:52893","req_id":"966b0f3a-fe98-4798-9e8b-68571a9358cd","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 23187,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 0,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"oxz_cockroachdb_54b02a38-ac6b-4866-bfe6-cc383b36872d.\"),\n ),\n original: Query {\n name: Name(\"oxz_cockroachdb_54b02a38-ac6b-4866-bfe6-cc383b36872d.\"),\n query_type: A,\n query_class: IN,\n },\n },\n original: [\n 52,\n 111,\n 120,\n 122,\n 95,\n 99,\n 111,\n 99,\n 107,\n 114,\n 111,\n 97,\n 99,\n 104,\n 100,\n 98,\n 95,\n 53,\n 52,\n 98,\n 48,\n 50,\n 97,\n 51,\n 56,\n 45,\n 97,\n 99,\n 54,\n 98,\n 45,\n 52,\n 56,\n 54,\n 54,\n 45,\n 98,\n 102,\n 101,\n 54,\n 45,\n 99,\n 99,\n 51,\n 56,\n 51,\n 98,\n 51,\n 54,\n 56,\n 55,\n 50,\n 100,\n 0,\n 0,\n 1,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: None,\n} SERVFAIL: server is not authoritative for name: \"oxz_cockroachdb_54b02a38-ac6b-4866-bfe6-cc383b36872d.\"","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:30:01.027701774Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::6]:50437","req_id":"92b05010-8510-4d2e-bb8b-a5b1bfbaf902","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 30288,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:30:09.332240075Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:52957","req_id":"c33b4f86-629b-4e41-b77c-22b855493cc1","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 18180,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:30:19.333225336Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:58980","req_id":"a8e5e430-256d-4d26-9a69-eafd71d27751","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:28.066713459Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:61723"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:28.066931355Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"cc967ba1-076b-49e9-8b16-0c978ccbdbc5","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:28.066942436Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"cc967ba1-076b-49e9-8b16-0c978ccbdbc5","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:28.066950256Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"uri":"/config","method":"PUT","req_id":"cc967ba1-076b-49e9-8b16-0c978ccbdbc5","remote_addr":"[fd00:1122:3344:101::b]:61723","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":165,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 37602,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:30:29.334778782Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:59835","req_id":"90561087-b548-4200-97a8-66113c7febba","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:29.570723085Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:34909"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:29.570970781Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"1c152c8f-21b0-4e5d-bd7d-61f27fff0157","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:29.570988902Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"1c152c8f-21b0-4e5d-bd7d-61f27fff0157","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:29.570997442Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"uri":"/config","method":"PUT","req_id":"1c152c8f-21b0-4e5d-bd7d-61f27fff0157","remote_addr":"[fd00:1122:3344:101::c]:34909","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":194,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:29.778377933Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:47354"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:29.778586769Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"21044a48-3efa-4c47-87dd-e61a8d88efff","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:29.778599519Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"21044a48-3efa-4c47-87dd-e61a8d88efff","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:30:29.778607309Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"uri":"/config","method":"PUT","req_id":"21044a48-3efa-4c47-87dd-e61a8d88efff","remote_addr":"[fd00:1122:3344:101::a]:47354","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":171,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 58066,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:30:39.335811856Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:53041","req_id":"a40f77e8-87a3-481c-9f37-af41335400f2","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 58961,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:30:49.337867066Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:64930","req_id":"a12efcf5-e916-46bd-bed5-d8873cd73ad9","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 42398,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:30:59.339709839Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:63602","req_id":"e19e6786-b51e-4cdd-a450-7903162643e7","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 45601,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:31:09.34126976Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:49936","req_id":"dfcb33c4-7d92-4c59-a38c-a4af8624b6d6","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 22919,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:31:19.342767411Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:56735","req_id":"f96352ce-1493-4bfd-8964-22ce38e9bc29","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:28.117082583Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::b]:51273"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:28.11732537Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"db690c34-2d17-4a84-b48b-4d8d9a3d1f71","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:28.11733926Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"db690c34-2d17-4a84-b48b-4d8d9a3d1f71","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:28.11734657Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"uri":"/config","method":"PUT","req_id":"db690c34-2d17-4a84-b48b-4d8d9a3d1f71","remote_addr":"[fd00:1122:3344:101::b]:51273","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":190,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 13338,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:31:29.343961402Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:64065","req_id":"f69e5dab-fb0b-4d8f-8196-323d991f450b","component":"dns"} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:29.451928483Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::c]:60776"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:29.452222091Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"3ed717a5-5a60-46c1-859e-40c6a3ff5b75","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:29.452240392Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"3ed717a5-5a60-46c1-859e-40c6a3ff5b75","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:29.452249382Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"uri":"/config","method":"PUT","req_id":"3ed717a5-5a60-46c1-859e-40c6a3ff5b75","remote_addr":"[fd00:1122:3344:101::c]:60776","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":219,"response_code":204} {"msg":"accepted connection","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:29.843928807Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","remote_addr":"[fd00:1122:3344:101::a]:42273"} {"msg":"attempting generation update","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:29.844120503Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"6fcb3b2f-39a3-4f33-afed-bd9e8bad4690","component":"store"} {"msg":"updated generation","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:29.844133843Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"new_generation":1,"req_id":"6fcb3b2f-39a3-4f33-afed-bd9e8bad4690","component":"store"} {"msg":"request completed","v":0,"name":"dns-server","level":30,"time":"2025-06-04T01:31:29.844145803Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"uri":"/config","method":"PUT","req_id":"6fcb3b2f-39a3-4f33-afed-bd9e8bad4690","remote_addr":"[fd00:1122:3344:101::a]:42273","local_addr":"[fd00:1122:3344:3::1]:5353","component":"http","latency_us":163,"response_code":204} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 10001,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:31:39.345396444Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:51099","req_id":"0f2e5160-3ffa-44ea-85e6-b2191843fbfc","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 40436,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:31:49.346545875Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:53348","req_id":"3c264ed3-eea1-465d-9eb9-079d17707a57","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 26647,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:31:59.348220315Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:59441","req_id":"9550b042-ec3b-41ff-8ec2-5dbd821db9a7","component":"dns"} {"msg":"failed to handle incoming DNS message: MessageRequest {\n header: Header {\n id: 46858,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n ),\n original: Query {\n name: Name(\"_clickhouse-cluster-native._tcp.control-plane.oxide.internal.\"),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 26,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 99,\n 108,\n 117,\n 115,\n 116,\n 101,\n 114,\n 45,\n 110,\n 97,\n 116,\n 105,\n 118,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal","v":0,"name":"dns-server","level":50,"time":"2025-06-04T01:32:09.350080028Z","hostname":"oxz_internal_dns_aacbc88f-98f9-423b-9e93-03d859fe410d","pid":5183,"peer_addr":"[fd00:1122:3344:101::d]:59528","req_id":"460a687e-7a81-48e2-b92e-3ac713b69744","component":"dns"}