Max level shown:
1 [ Jul 16 13:11:11 Enabled. ]
2 [ Jul 16 13:11:11 Rereading configuration. ]
3 [ Jul 16 13:11:11 Rereading configuration. ]
4 [ Jul 16 13:11:13 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dns-server/bin/dns-server --config-file /var/svc/manifest/site/internal_dns/config.toml --http-address [fd00:1122:3344:2::1]:5353 --dns-address [fd00:1122:3344:2::1]:53 &"). ]
5 [ Jul 16 13:11:13 Method "start" exited with status 0. ]
6 Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:2::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } }
7 note: configured to log to "/dev/stdout"
82026-07-16T13:11:13.268ZINFOdns-server: config config = Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:2::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } } dns_config = Config { bind_address: [fd00:1122:3344:2::1]:53 }
92026-07-16T13:11:13.268ZINFOdns-server (store): opening sled database path = /data/dns
102026-07-16T13:11:13.290ZINFOdns-server (store): pruning trees for generations newer than 0
112026-07-16T13:11:13.290ZINFOdns-server (store): pruning trees for generations older than 0 keep = 3
122026-07-16T13:11:13.290ZINFOdns-server (dns): DNS server bound to address local_address = [fd00:1122:3344:2::1]:53
132026-07-16T13:11:13.290ZINFOdns-server (http): listening local_addr = [fd00:1122:3344:2::1]:5353
142026-07-16T13:11:14.705ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:2::2]:64877
152026-07-16T13:11:14.706ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = e8198277-88c9-4bf9-8d5c-51f8dda4c0be
162026-07-16T13:11:14.706ZINFOdns-server (store): pruning trees for generations newer than 0
172026-07-16T13:11:14.737ZINFOdns-server (store): pruning trees for generations older than 1 keep = 3
182026-07-16T13:11:14.737ZINFOdns-server (store): updated generation new_generation = 1 req_id = e8198277-88c9-4bf9-8d5c-51f8dda4c0be
192026-07-16T13:11:14.737ZINFOdns-server (http): request completed latency_us = 31621 local_addr = [fd00:1122:3344:2::1]:5353 method = PUT remote_addr = [fd00:1122:3344:2::2]:64877 req_id = e8198277-88c9-4bf9-8d5c-51f8dda4c0be response_code = 204 uri = /config
202026-07-16T13:11:26.920ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11790, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:35836 req_id = 2e9304ff-b7b3-49f3-8fe2-656529cef7ec
212026-07-16T13:11:26.924ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54333, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:47953 req_id = 45131323-b596-4aca-887c-9bf3b8901a5c
222026-07-16T13:11:26.928ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37711, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:62446 req_id = b10e9281-be80-4701-b39c-0e64f39ea227
232026-07-16T13:12:18.558ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32155, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:49640 req_id = 5fdbde49-4353-46e2-9e93-de71bee5dfe3
242026-07-16T13:12:18.562ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 30940, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:53748 req_id = d83ef3b3-1692-4425-97b7-a26d4308b063
252026-07-16T13:12:18.567ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 58055, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:61053 req_id = dbc22b83-e1e1-41f4-a864-229a11687efd
262026-07-16T13:14:02.626ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 23683, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:64981 req_id = 2ed16901-f3a3-43d5-bc19-a643ade14f41
272026-07-16T13:14:11.555ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 29965, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::b]:46515 req_id = 2105b42d-5cbb-4896-b065-bca2f13132db
282026-07-16T13:14:11.580ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5111, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:62283 req_id = fc46abb0-2387-4d20-80c1-01fe9582be0f
292026-07-16T13:14:12.608ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31300, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:49563 req_id = c9733e8e-11b5-4508-8901-943ffa43a61a
302026-07-16T13:14:12.902ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61160, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:40778 req_id = 495ad858-6bdb-47f4-a306-2daae6a30f15
312026-07-16T13:14:12.941ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 29623, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2."), ), original: Query { name: Name("oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 48, 98, 100, 50, 99, 52, 49, 100, 45, 50, 100, 52, 54, 45, 52, 49, 48, 98, 45, 98, 54, 49, 100, 45, 98, 55, 50, 98, 57, 54, 57, 52, 49, 51, 100, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2." peer_addr = [fd00:1122:3344:101::e]:51727 req_id = 7ae86c73-36f6-4b35-ac2e-22c1e58ac064
322026-07-16T13:14:12.945ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 29623, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2."), ), original: Query { name: Name("oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 48, 98, 100, 50, 99, 52, 49, 100, 45, 50, 100, 52, 54, 45, 52, 49, 48, 98, 45, 98, 54, 49, 100, 45, 98, 55, 50, 98, 57, 54, 57, 52, 49, 51, 100, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2." peer_addr = [fd00:1122:3344:101::e]:45627 req_id = 57b27fce-05b8-42d5-827c-a3dd06ff27f7
332026-07-16T13:14:22.610ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54175, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:63440 req_id = 0302b709-e1ea-4265-b668-94d5d0d8281a
342026-07-16T13:14:30.126ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 9421, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:47890 req_id = b1d3d2e9-18b6-4554-969e-7d5a21c48b0f
352026-07-16T13:14:30.485ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::a]:65535
362026-07-16T13:14:30.529ZINFOdns-server (http): request completed latency_us = 21321 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:65535 req_id = 3b4bfa50-900e-4104-9435-a7ce4f168766 response_code = 200 uri = /config
372026-07-16T13:14:32.611ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 58078, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:50792 req_id = 3c5b3323-0996-4f77-872b-93da4bc098ed
382026-07-16T13:14:33.093ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::c]:52972
392026-07-16T13:14:33.099ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 62017, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:4916 req_id = 99a24cff-c323-4ef7-8ca1-3ffe5419afc5
402026-07-16T13:14:33.101ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 7993fd89-68e4-4358-9e86-4fc32653ee0f
412026-07-16T13:14:33.101ZINFOdns-server (store): updated generation new_generation = 1 req_id = 7993fd89-68e4-4358-9e86-4fc32653ee0f
422026-07-16T13:14:33.101ZINFOdns-server (http): request completed latency_us = 7213 local_addr = [fd00:1122:3344:2::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::c]:52972 req_id = 7993fd89-68e4-4358-9e86-4fc32653ee0f response_code = 204 uri = /config
432026-07-16T13:14:33.244ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::c]:47599
442026-07-16T13:14:33.245ZINFOdns-server (http): request completed latency_us = 199 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:47599 req_id = 0257911e-270a-4c35-8ead-c2eb33c850af response_code = 200 uri = /config
452026-07-16T13:14:33.949ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52911, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:52222 req_id = 7fa89f5f-908c-4885-878c-f1d2f8dfe086
462026-07-16T13:14:34.074ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::c]:51881
472026-07-16T13:14:34.074ZINFOdns-server (http): request completed latency_us = 239 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:51881 req_id = e9bf9b29-4cda-439f-9364-ec2e308a515f response_code = 200 uri = /config
482026-07-16T13:14:42.366ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 46663, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:56779 req_id = 98fe9cd2-4246-4d55-8dd0-d9251ac5783c
492026-07-16T13:14:42.403ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 56235, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::b]:50623 req_id = 0f4c7be3-119d-4829-83b9-a122d14c91d3
502026-07-16T13:14:42.490ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::a]:57463
512026-07-16T13:14:42.490ZINFOdns-server (http): request completed latency_us = 196 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:57463 req_id = 278dbf45-5001-4513-aed4-17509c9e821b response_code = 200 uri = /config
522026-07-16T13:14:42.613ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59028, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:65178 req_id = 90469321-a874-4595-8717-e8ef8efdf73a
532026-07-16T13:14:42.626ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::b]:35699
542026-07-16T13:14:42.627ZINFOdns-server (http): request completed latency_us = 231 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:35699 req_id = ac99e305-e4a6-4b8e-94a2-41806f34e57e response_code = 200 uri = /config
552026-07-16T13:14:52.615ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 28111, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:49475 req_id = 7ec0f7c2-a7d4-4bcc-848a-b0f658265bbc
562026-07-16T13:15:00.039ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 28209, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_6db2db7d-49e9-4fac-abb1-9c716ad09c04."), ), original: Query { name: Name("oxz_cockroachdb_6db2db7d-49e9-4fac-abb1-9c716ad09c04."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 54, 100, 98, 50, 100, 98, 55, 100, 45, 52, 57, 101, 57, 45, 52, 102, 97, 99, 45, 97, 98, 98, 49, 45, 57, 99, 55, 49, 54, 97, 100, 48, 57, 99, 48, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_6db2db7d-49e9-4fac-abb1-9c716ad09c04." peer_addr = [fd00:1122:3344:101::5]:63290 req_id = c36bcf1c-edce-4019-80e6-7678a009bf73
572026-07-16T13:15:00.040ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 28209, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_6db2db7d-49e9-4fac-abb1-9c716ad09c04."), ), original: Query { name: Name("oxz_cockroachdb_6db2db7d-49e9-4fac-abb1-9c716ad09c04."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 54, 100, 98, 50, 100, 98, 55, 100, 45, 52, 57, 101, 57, 45, 52, 102, 97, 99, 45, 97, 98, 98, 49, 45, 57, 99, 55, 49, 54, 97, 100, 48, 57, 99, 48, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_6db2db7d-49e9-4fac-abb1-9c716ad09c04." peer_addr = [fd00:1122:3344:101::5]:51048 req_id = ee3f3591-28fd-4ca0-85e7-4f222c5ead4a
582026-07-16T13:15:00.260ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6023, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_ntp_cf173f09-d8d0-4b1e-b24c-99bd462477bd."), ), original: Query { name: Name("oxz_ntp_cf173f09-d8d0-4b1e-b24c-99bd462477bd."), query_type: A, query_class: IN, }, }, ], original: [ 44, 111, 120, 122, 95, 110, 116, 112, 95, 99, 102, 49, 55, 51, 102, 48, 57, 45, 100, 56, 100, 48, 45, 52, 98, 49, 101, 45, 98, 50, 52, 99, 45, 57, 57, 98, 100, 52, 54, 50, 52, 55, 55, 98, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_ntp_cf173f09-d8d0-4b1e-b24c-99bd462477bd." peer_addr = [fd00:1122:3344:101::1b]:52896 req_id = 838a6510-9dd3-4ebd-87ae-0579e6a93d61
592026-07-16T13:15:00.613ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 25735, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_8fc522e2-5b81-4ed8-b38c-f0c270db5566."), ), original: Query { name: Name("oxz_cockroachdb_8fc522e2-5b81-4ed8-b38c-f0c270db5566."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 56, 102, 99, 53, 50, 50, 101, 50, 45, 53, 98, 56, 49, 45, 52, 101, 100, 56, 45, 98, 51, 56, 99, 45, 102, 48, 99, 50, 55, 48, 100, 98, 53, 53, 54, 54, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_8fc522e2-5b81-4ed8-b38c-f0c270db5566." peer_addr = [fd00:1122:3344:101::4]:47236 req_id = 6ed41703-75c7-4932-bbd3-af61073be645
602026-07-16T13:15:00.614ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 25735, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_8fc522e2-5b81-4ed8-b38c-f0c270db5566."), ), original: Query { name: Name("oxz_cockroachdb_8fc522e2-5b81-4ed8-b38c-f0c270db5566."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 56, 102, 99, 53, 50, 50, 101, 50, 45, 53, 98, 56, 49, 45, 52, 101, 100, 56, 45, 98, 51, 56, 99, 45, 102, 48, 99, 50, 55, 48, 100, 98, 53, 53, 54, 54, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_8fc522e2-5b81-4ed8-b38c-f0c270db5566." peer_addr = [fd00:1122:3344:101::4]:58769 req_id = 37ac1f41-f1e9-40b8-af6d-c611c964fb4a
612026-07-16T13:15:00.629ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19634, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2."), ), original: Query { name: Name("oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 48, 98, 100, 50, 99, 52, 49, 100, 45, 50, 100, 52, 54, 45, 52, 49, 48, 98, 45, 98, 54, 49, 100, 45, 98, 55, 50, 98, 57, 54, 57, 52, 49, 51, 100, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2." peer_addr = [fd00:1122:3344:101::e]:34394 req_id = e737e8eb-c2b7-42d1-95c2-a13aa364eab4
622026-07-16T13:15:00.630ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19634, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2."), ), original: Query { name: Name("oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 48, 98, 100, 50, 99, 52, 49, 100, 45, 50, 100, 52, 54, 45, 52, 49, 48, 98, 45, 98, 54, 49, 100, 45, 98, 55, 50, 98, 57, 54, 57, 52, 49, 51, 100, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_0bd2c41d-2d46-410b-b61d-b72b969413d2." peer_addr = [fd00:1122:3344:101::e]:40930 req_id = d9f32a31-423c-4872-903e-b9ba0fa82d1b
632026-07-16T13:15:00.701ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 4607, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." peer_addr = [fd00:1122:3344:101::2]:59408 req_id = f623cbc4-7d10-4556-857c-1851809b597c
642026-07-16T13:15:00.702ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 4607, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." peer_addr = [fd00:1122:3344:101::2]:41363 req_id = 8425e43f-8211-4fd6-9817-992de9ab3885
652026-07-16T13:15:00.738ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19867, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_c0f58561-0d04-4c2c-8a1f-e928d3198b2f."), ), original: Query { name: Name("oxz_cockroachdb_c0f58561-0d04-4c2c-8a1f-e928d3198b2f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 99, 48, 102, 53, 56, 53, 54, 49, 45, 48, 100, 48, 52, 45, 52, 99, 50, 99, 45, 56, 97, 49, 102, 45, 101, 57, 50, 56, 100, 51, 49, 57, 56, 98, 50, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_c0f58561-0d04-4c2c-8a1f-e928d3198b2f." peer_addr = [fd00:1122:3344:101::7]:38335 req_id = 5fc2ad0d-1eec-45ce-94a2-36db973888fa
662026-07-16T13:15:00.738ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19867, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_c0f58561-0d04-4c2c-8a1f-e928d3198b2f."), ), original: Query { name: Name("oxz_cockroachdb_c0f58561-0d04-4c2c-8a1f-e928d3198b2f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 99, 48, 102, 53, 56, 53, 54, 49, 45, 48, 100, 48, 52, 45, 52, 99, 50, 99, 45, 56, 97, 49, 102, 45, 101, 57, 50, 56, 100, 51, 49, 57, 56, 98, 50, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_c0f58561-0d04-4c2c-8a1f-e928d3198b2f." peer_addr = [fd00:1122:3344:101::7]:50452 req_id = 07e93b25-9df1-46fc-8ada-a85f96a01fac
672026-07-16T13:15:00.870ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31177, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_e383ae00-f4aa-4e94-b153-bfb927fb77d5."), ), original: Query { name: Name("oxz_cockroachdb_e383ae00-f4aa-4e94-b153-bfb927fb77d5."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 101, 51, 56, 51, 97, 101, 48, 48, 45, 102, 52, 97, 97, 45, 52, 101, 57, 52, 45, 98, 49, 53, 51, 45, 98, 102, 98, 57, 50, 55, 102, 98, 55, 55, 100, 53, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_e383ae00-f4aa-4e94-b153-bfb927fb77d5." peer_addr = [fd00:1122:3344:101::3]:50768 req_id = 82490d48-d024-4db3-ad3e-1a6bedd8cc66
682026-07-16T13:15:00.871ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31177, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_e383ae00-f4aa-4e94-b153-bfb927fb77d5."), ), original: Query { name: Name("oxz_cockroachdb_e383ae00-f4aa-4e94-b153-bfb927fb77d5."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 101, 51, 56, 51, 97, 101, 48, 48, 45, 102, 52, 97, 97, 45, 52, 101, 57, 52, 45, 98, 49, 53, 51, 45, 98, 102, 98, 57, 50, 55, 102, 98, 55, 55, 100, 53, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_e383ae00-f4aa-4e94-b153-bfb927fb77d5." peer_addr = [fd00:1122:3344:101::3]:57567 req_id = 00a96c64-db03-42fb-bfd6-9eaccd797eba
692026-07-16T13:15:01.002ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 13061, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_3faab002-3293-4d82-aef2-1a26645f4d3d."), ), original: Query { name: Name("oxz_cockroachdb_3faab002-3293-4d82-aef2-1a26645f4d3d."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 51, 102, 97, 97, 98, 48, 48, 50, 45, 51, 50, 57, 51, 45, 52, 100, 56, 50, 45, 97, 101, 102, 50, 45, 49, 97, 50, 54, 54, 52, 53, 102, 52, 100, 51, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_3faab002-3293-4d82-aef2-1a26645f4d3d." peer_addr = [fd00:1122:3344:101::6]:40746 req_id = 5af8e2bb-3690-437e-a675-701d39aa96a5
702026-07-16T13:15:01.003ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 13061, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_3faab002-3293-4d82-aef2-1a26645f4d3d."), ), original: Query { name: Name("oxz_cockroachdb_3faab002-3293-4d82-aef2-1a26645f4d3d."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 51, 102, 97, 97, 98, 48, 48, 50, 45, 51, 50, 57, 51, 45, 52, 100, 56, 50, 45, 97, 101, 102, 50, 45, 49, 97, 50, 54, 54, 52, 53, 102, 52, 100, 51, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_3faab002-3293-4d82-aef2-1a26645f4d3d." peer_addr = [fd00:1122:3344:101::6]:57076 req_id = 723b023f-f491-47df-a838-92c09a280551