Max level shown:
1 [ Jul 16 22:57:19 Enabled. ]
2 [ Jul 16 22:57:19 Rereading configuration. ]
3 [ Jul 16 22:57:20 Rereading configuration. ]
4 [ Jul 16 22:57:21 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dns-server/bin/dns-server --config-file /var/svc/manifest/site/internal_dns/config.toml --http-address [fd00:1122:3344:1::1]:5353 --dns-address [fd00:1122:3344:1::1]:53 &"). ]
5 [ Jul 16 22:57:21 Method "start" exited with status 0. ]
6 Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:1::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } }
7 note: configured to log to "/dev/stdout"
82026-07-16T22:57:21.666ZINFOdns-server: config config = Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:1::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } } dns_config = Config { bind_address: [fd00:1122:3344:1::1]:53 }
92026-07-16T22:57:21.666ZINFOdns-server (store): opening sled database path = /data/dns
102026-07-16T22:57:21.688ZINFOdns-server (store): pruning trees for generations newer than 0
112026-07-16T22:57:21.688ZINFOdns-server (store): pruning trees for generations older than 0 keep = 3
122026-07-16T22:57:21.688ZINFOdns-server (dns): DNS server bound to address local_address = [fd00:1122:3344:1::1]:53
132026-07-16T22:57:21.688ZINFOdns-server (http): listening local_addr = [fd00:1122:3344:1::1]:5353
142026-07-16T22:57:21.735ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:1::2]:54495
152026-07-16T22:57:21.736ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = f6854389-81b4-4662-bcf8-fdae2c792a21
162026-07-16T22:57:21.736ZINFOdns-server (store): pruning trees for generations newer than 0
172026-07-16T22:57:21.748ZINFOdns-server (store): pruning trees for generations older than 1 keep = 3
182026-07-16T22:57:21.748ZINFOdns-server (store): updated generation new_generation = 1 req_id = f6854389-81b4-4662-bcf8-fdae2c792a21
192026-07-16T22:57:21.748ZINFOdns-server (http): request completed latency_us = 13192 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:1::2]:54495 req_id = f6854389-81b4-4662-bcf8-fdae2c792a21 response_code = 204 uri = /config
202026-07-16T22:57:33.823ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5421, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:44381 req_id = eee1b622-cc00-4e53-bb22-4f751d3a23a7
212026-07-16T22:57:33.828ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 672, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:48894 req_id = 20b0b9f0-e7d0-4590-8355-74d3172160ea
222026-07-16T22:57:33.832ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31222, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:62778 req_id = ebd548ae-5b5e-437d-ab47-78f865061fff
232026-07-16T22:58:42.881ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 20205, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:50400 req_id = 7a5a8c98-b087-4652-8bf6-b968466d3afc
242026-07-16T22:58:46.254ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 25732, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522."), ), original: Query { name: Name("oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 98, 101, 101, 99, 97, 99, 51, 48, 45, 54, 98, 53, 102, 45, 52, 98, 97, 52, 45, 56, 48, 50, 56, 45, 48, 53, 97, 53, 100, 55, 100, 50, 57, 53, 50, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522." peer_addr = [fd00:1122:3344:101::e]:36503 req_id = 6bd7660c-d5fb-4b73-af5f-b8edbf769cbf
252026-07-16T22:58:46.262ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 25732, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522."), ), original: Query { name: Name("oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 98, 101, 101, 99, 97, 99, 51, 48, 45, 54, 98, 53, 102, 45, 52, 98, 97, 52, 45, 56, 48, 50, 56, 45, 48, 53, 97, 53, 100, 55, 100, 50, 57, 53, 50, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522." peer_addr = [fd00:1122:3344:101::e]:45234 req_id = 9788ac0c-955c-4ef0-a09d-7940381e27da
262026-07-16T22:58:51.220ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18121, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::b]:41414 req_id = 2c43db8e-5677-42eb-a4ab-33a06d0d0aab
272026-07-16T22:58:51.562ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55387, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:13692 req_id = 007adc47-2e1f-4607-bd36-1051d362a86b
282026-07-16T22:58:52.139ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 29078, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:24195 req_id = 0c72cd5a-cd77-46f6-b5ec-3ecee991fa4f
292026-07-16T22:58:52.858ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38691, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:59912 req_id = 3ca8ca58-d73f-49b3-bca4-4a39e005c118
302026-07-16T22:59:02.860ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31204, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:54683 req_id = de5de348-4c81-418d-97e1-97f0673a7700
312026-07-16T22:59:12.861ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 40928, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:62479 req_id = f5e2ead1-ba08-42ab-b36b-2c46758294e0
322026-07-16T22:59:22.862ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 21211, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:50223 req_id = 00237965-8fe5-42ee-82d7-a966521f7f38
332026-07-16T22:59:32.864ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61142, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:57018 req_id = d5d2aa33-b191-4ac0-8c50-2f827c555df2
342026-07-16T22:59:40.771ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 10454, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:63602 req_id = 246c1b8e-77c9-41dc-a9cf-9512e6daa34d
352026-07-16T22:59:41.048ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:38965
362026-07-16T22:59:41.104ZINFOdns-server (http): request completed latency_us = 37528 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:38965 req_id = 2c572615-bd0c-4c46-a1a2-7f88c2b71eee response_code = 200 uri = /config
372026-07-16T22:59:42.865ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 36032, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:61879 req_id = 2b65ae75-5a42-4c11-918a-2caf412f703f
382026-07-16T22:59:43.766ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 43504, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:35673 req_id = 6a4708d9-dd3e-47c9-ad96-c701e9720375
392026-07-16T22:59:43.904ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:33913
402026-07-16T22:59:43.910ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 767d0232-9d52-4454-8dbc-5ab0b95fe177
412026-07-16T22:59:43.910ZINFOdns-server (store): updated generation new_generation = 1 req_id = 767d0232-9d52-4454-8dbc-5ab0b95fe177
422026-07-16T22:59:43.910ZINFOdns-server (http): request completed latency_us = 3208 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::c]:33913 req_id = 767d0232-9d52-4454-8dbc-5ab0b95fe177 response_code = 204 uri = /config
432026-07-16T22:59:43.976ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:43523
442026-07-16T22:59:43.976ZINFOdns-server (http): request completed latency_us = 287 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:43523 req_id = d0927f2d-3955-4009-9706-dd5d17e9e767 response_code = 200 uri = /config
452026-07-16T22:59:44.851ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 51481, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:19217 req_id = 8e51e931-e687-4053-9b5e-be0412250f86
462026-07-16T22:59:44.945ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:34164
472026-07-16T22:59:44.945ZINFOdns-server (http): request completed latency_us = 264 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:34164 req_id = e26713e0-4231-4470-bc1d-9bdc0eb92013 response_code = 200 uri = /config
482026-07-16T22:59:51.301ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:38634
492026-07-16T22:59:51.305ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = f4dce6bb-cbbc-4293-8146-67205673cd73
502026-07-16T22:59:51.305ZINFOdns-server (store): updated generation new_generation = 1 req_id = f4dce6bb-cbbc-4293-8146-67205673cd73
512026-07-16T22:59:51.305ZINFOdns-server (http): request completed latency_us = 228 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::b]:38634 req_id = f4dce6bb-cbbc-4293-8146-67205673cd73 response_code = 204 uri = /config
522026-07-16T22:59:51.373ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:39927
532026-07-16T22:59:51.375ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = b072c9e2-cae8-4509-9237-48f310e74948
542026-07-16T22:59:51.375ZINFOdns-server (store): updated generation new_generation = 1 req_id = b072c9e2-cae8-4509-9237-48f310e74948
552026-07-16T22:59:51.375ZINFOdns-server (http): request completed latency_us = 222 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::c]:39927 req_id = b072c9e2-cae8-4509-9237-48f310e74948 response_code = 204 uri = /config
562026-07-16T22:59:52.148ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:48871
572026-07-16T22:59:52.150ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 8c47ab0f-0991-4d2f-b156-8646f29fca31
582026-07-16T22:59:52.150ZINFOdns-server (store): updated generation new_generation = 1 req_id = 8c47ab0f-0991-4d2f-b156-8646f29fca31
592026-07-16T22:59:52.150ZINFOdns-server (http): request completed latency_us = 358 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::a]:48871 req_id = 8c47ab0f-0991-4d2f-b156-8646f29fca31 response_code = 204 uri = /config
602026-07-16T22:59:52.165ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 9439, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:50525 req_id = bcf49996-db80-4143-995a-4b5237c962d3
612026-07-16T22:59:52.243ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 42800, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::b]:13992 req_id = d6334f69-3809-493f-8c18-db347ac3eb57
622026-07-16T22:59:52.313ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:38235
632026-07-16T22:59:52.313ZINFOdns-server (http): request completed latency_us = 372 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:38235 req_id = af517dc9-6e5d-45cf-a042-540a109206fc response_code = 200 uri = /config
642026-07-16T22:59:52.394ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:45061
652026-07-16T22:59:52.394ZINFOdns-server (http): request completed latency_us = 218 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:45061 req_id = c312126c-96dc-47ec-85e8-9a99dd87179b response_code = 200 uri = /config
662026-07-16T22:59:52.867ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 58981, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:61011 req_id = 6136f017-ad87-4fba-97cb-6e04b931dd05
672026-07-16T22:59:53.636ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 53486, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:22162 req_id = a3ccef87-5cfa-4879-a939-2c085c8b35da
682026-07-16T22:59:53.739ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:51604
692026-07-16T22:59:53.739ZINFOdns-server (http): request completed latency_us = 284 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:51604 req_id = 9265effe-3090-4de9-b80d-2969dda26a20 response_code = 200 uri = /config
702026-07-16T23:00:00.145ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35255, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_36d1dc0b-4a0e-4247-b4b2-dc06ec8d0916."), ), original: Query { name: Name("oxz_cockroachdb_36d1dc0b-4a0e-4247-b4b2-dc06ec8d0916."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 51, 54, 100, 49, 100, 99, 48, 98, 45, 52, 97, 48, 101, 45, 52, 50, 52, 55, 45, 98, 52, 98, 50, 45, 100, 99, 48, 54, 101, 99, 56, 100, 48, 57, 49, 54, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_36d1dc0b-4a0e-4247-b4b2-dc06ec8d0916." peer_addr = [fd00:1122:3344:101::4]:61819 req_id = e631a152-b14b-4896-a91a-d08fc959f4c1
712026-07-16T23:00:00.146ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35255, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_36d1dc0b-4a0e-4247-b4b2-dc06ec8d0916."), ), original: Query { name: Name("oxz_cockroachdb_36d1dc0b-4a0e-4247-b4b2-dc06ec8d0916."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 51, 54, 100, 49, 100, 99, 48, 98, 45, 52, 97, 48, 101, 45, 52, 50, 52, 55, 45, 98, 52, 98, 50, 45, 100, 99, 48, 54, 101, 99, 56, 100, 48, 57, 49, 54, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_36d1dc0b-4a0e-4247-b4b2-dc06ec8d0916." peer_addr = [fd00:1122:3344:101::4]:58356 req_id = d130bbac-b855-4d60-88aa-3a003d84a107
722026-07-16T23:00:00.158ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54376, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_58ea2790-4a1f-4cf2-96f0-e47d97c13dcc."), ), original: Query { name: Name("oxz_cockroachdb_58ea2790-4a1f-4cf2-96f0-e47d97c13dcc."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 53, 56, 101, 97, 50, 55, 57, 48, 45, 52, 97, 49, 102, 45, 52, 99, 102, 50, 45, 57, 54, 102, 48, 45, 101, 52, 55, 100, 57, 55, 99, 49, 51, 100, 99, 99, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_58ea2790-4a1f-4cf2-96f0-e47d97c13dcc." peer_addr = [fd00:1122:3344:101::6]:41562 req_id = 6309413f-d7a3-4334-b339-9f8224276fcb
732026-07-16T23:00:00.159ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54376, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_58ea2790-4a1f-4cf2-96f0-e47d97c13dcc."), ), original: Query { name: Name("oxz_cockroachdb_58ea2790-4a1f-4cf2-96f0-e47d97c13dcc."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 53, 56, 101, 97, 50, 55, 57, 48, 45, 52, 97, 49, 102, 45, 52, 99, 102, 50, 45, 57, 54, 102, 48, 45, 101, 52, 55, 100, 57, 55, 99, 49, 51, 100, 99, 99, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_58ea2790-4a1f-4cf2-96f0-e47d97c13dcc." peer_addr = [fd00:1122:3344:101::6]:43351 req_id = bbd0db7c-de27-4a38-8ce6-ef8050e02c5f
742026-07-16T23:00:00.160ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54777, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_5bbe5275-caa2-4fb0-b6a7-38d98b53802a."), ), original: Query { name: Name("oxz_cockroachdb_5bbe5275-caa2-4fb0-b6a7-38d98b53802a."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 53, 98, 98, 101, 53, 50, 55, 53, 45, 99, 97, 97, 50, 45, 52, 102, 98, 48, 45, 98, 54, 97, 55, 45, 51, 56, 100, 57, 56, 98, 53, 51, 56, 48, 50, 97, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_5bbe5275-caa2-4fb0-b6a7-38d98b53802a." peer_addr = [fd00:1122:3344:101::7]:36414 req_id = 0da95e4e-0d8e-4a9c-969e-993bd4ed5751
752026-07-16T23:00:00.161ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54777, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_5bbe5275-caa2-4fb0-b6a7-38d98b53802a."), ), original: Query { name: Name("oxz_cockroachdb_5bbe5275-caa2-4fb0-b6a7-38d98b53802a."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 53, 98, 98, 101, 53, 50, 55, 53, 45, 99, 97, 97, 50, 45, 52, 102, 98, 48, 45, 98, 54, 97, 55, 45, 51, 56, 100, 57, 56, 98, 53, 51, 56, 48, 50, 97, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_5bbe5275-caa2-4fb0-b6a7-38d98b53802a." peer_addr = [fd00:1122:3344:101::7]:51722 req_id = 70046a53-3da4-423e-82ca-bb159c57a085
762026-07-16T23:00:00.193ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61673, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_93ba913b-a291-439b-8085-613a88dfb140."), ), original: Query { name: Name("oxz_cockroachdb_93ba913b-a291-439b-8085-613a88dfb140."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 51, 98, 97, 57, 49, 51, 98, 45, 97, 50, 57, 49, 45, 52, 51, 57, 98, 45, 56, 48, 56, 53, 45, 54, 49, 51, 97, 56, 56, 100, 102, 98, 49, 52, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_93ba913b-a291-439b-8085-613a88dfb140." peer_addr = [fd00:1122:3344:101::3]:49901 req_id = 703d8c2a-d737-4939-81fb-b2a4639b7c60
772026-07-16T23:00:00.193ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61673, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_93ba913b-a291-439b-8085-613a88dfb140."), ), original: Query { name: Name("oxz_cockroachdb_93ba913b-a291-439b-8085-613a88dfb140."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 51, 98, 97, 57, 49, 51, 98, 45, 97, 50, 57, 49, 45, 52, 51, 57, 98, 45, 56, 48, 56, 53, 45, 54, 49, 51, 97, 56, 56, 100, 102, 98, 49, 52, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_93ba913b-a291-439b-8085-613a88dfb140." peer_addr = [fd00:1122:3344:101::3]:47891 req_id = aebd86b1-c47a-4fb9-8b3e-26aaf39aaf7a
782026-07-16T23:00:00.256ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55664, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_9d6def0a-826e-4ffe-9a15-dfa058b5169b."), ), original: Query { name: Name("oxz_cockroachdb_9d6def0a-826e-4ffe-9a15-dfa058b5169b."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 100, 54, 100, 101, 102, 48, 97, 45, 56, 50, 54, 101, 45, 52, 102, 102, 101, 45, 57, 97, 49, 53, 45, 100, 102, 97, 48, 53, 56, 98, 53, 49, 54, 57, 98, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_9d6def0a-826e-4ffe-9a15-dfa058b5169b." peer_addr = [fd00:1122:3344:101::5]:38425 req_id = 459c9f02-fb4e-4b20-b872-f7f309f1c726
792026-07-16T23:00:00.257ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55664, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_9d6def0a-826e-4ffe-9a15-dfa058b5169b."), ), original: Query { name: Name("oxz_cockroachdb_9d6def0a-826e-4ffe-9a15-dfa058b5169b."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 100, 54, 100, 101, 102, 48, 97, 45, 56, 50, 54, 101, 45, 52, 102, 102, 101, 45, 57, 97, 49, 53, 45, 100, 102, 97, 48, 53, 56, 98, 53, 49, 54, 57, 98, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_9d6def0a-826e-4ffe-9a15-dfa058b5169b." peer_addr = [fd00:1122:3344:101::5]:46858 req_id = dcd64df4-2ad3-4391-81f3-f343688b4e18
802026-07-16T23:00:00.655ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3105, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." peer_addr = [fd00:1122:3344:101::2]:53971 req_id = 3699c546-1503-4564-9506-7468c105a6ac
812026-07-16T23:00:00.656ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3105, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." peer_addr = [fd00:1122:3344:101::2]:52236 req_id = dbc561ed-95e1-4b4b-b663-3a0c14ed6777
822026-07-16T23:00:00.822ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 30162, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_ntp_7d3994a0-53e1-4d66-81bc-e07591448165."), ), original: Query { name: Name("oxz_ntp_7d3994a0-53e1-4d66-81bc-e07591448165."), query_type: A, query_class: IN, }, }, ], original: [ 44, 111, 120, 122, 95, 110, 116, 112, 95, 55, 100, 51, 57, 57, 52, 97, 48, 45, 53, 51, 101, 49, 45, 52, 100, 54, 54, 45, 56, 49, 98, 99, 45, 101, 48, 55, 53, 57, 49, 52, 52, 56, 49, 54, 53, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_ntp_7d3994a0-53e1-4d66-81bc-e07591448165." peer_addr = [fd00:1122:3344:101::1b]:54547 req_id = 787a1ffd-fa55-4d8f-b980-995fb9f7ba4e
832026-07-16T23:00:01.003ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52847, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522."), ), original: Query { name: Name("oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 98, 101, 101, 99, 97, 99, 51, 48, 45, 54, 98, 53, 102, 45, 52, 98, 97, 52, 45, 56, 48, 50, 56, 45, 48, 53, 97, 53, 100, 55, 100, 50, 57, 53, 50, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522." peer_addr = [fd00:1122:3344:101::e]:34823 req_id = 3661ff89-3521-4574-97d7-5142695793da
842026-07-16T23:00:01.004ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52847, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522."), ), original: Query { name: Name("oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 98, 101, 101, 99, 97, 99, 51, 48, 45, 54, 98, 53, 102, 45, 52, 98, 97, 52, 45, 56, 48, 50, 56, 45, 48, 53, 97, 53, 100, 55, 100, 50, 57, 53, 50, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_beecac30-6b5f-4ba4-8028-05a5d7d29522." peer_addr = [fd00:1122:3344:101::e]:53043 req_id = 2cc10d70-b2f5-4529-bfd5-74a16ac59de3