Max level shown:
1 [ Oct 22 19:56:58 Enabled. ]
2 [ Oct 22 19:56:58 Rereading configuration. ]
3 [ Oct 22 19:56:58 Rereading configuration. ]
4 [ Oct 22 19:56:59 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dns-server/bin/dns-server --config-file /var/svc/manifest/site/internal_dns/config.toml --http-address [fd00:1122:3344:3::1]:5353 --dns-addresses [fd00:1122:3344:3::1]:53 &"). ]
5 [ Oct 22 19:56:59 Method "start" exited with status 0. ]
6 Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:3::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } }
7 note: configured to log to "/dev/stdout"
82026-10-22T19:56:59.984ZINFOdns-server: config config = Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:3::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } } dns_config = Config { bind_addresses: [[fd00:1122:3344:3::1]:53] }
92026-10-22T19:56:59.984ZINFOdns-server (store): opening sled database path = /data/dns
102026-10-22T19:57:00.007ZINFOdns-server (store): pruning trees for generations newer than 0
112026-10-22T19:57:00.007ZINFOdns-server (store): pruning trees for generations older than 0 keep = 3
122026-10-22T19:57:00.007ZINFOdns-server (dns): DNS server bound to address bind_address = [fd00:1122:3344:3::1]:53
132026-10-22T19:57:00.007ZINFOdns-server (http): listening local_addr = [fd00:1122:3344:3::1]:5353
142026-10-22T19:57:00.122ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22919, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::2]:37513 req_id = a000395b-cf42-4ab2-a8a4-f9e89a534b87
152026-10-22T19:57:00.276ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 49473, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::2]:5491 req_id = 41ad7424-2124-4a11-9b3f-d5cca7b5931f
162026-10-22T19:57:00.384ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19257, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::2]:33804 req_id = b024edaf-464e-4688-b4e7-b60062fceed6
172026-10-22T19:57:00.504ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:3::2]:59356
182026-10-22T19:57:00.505ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 0644d019-1ddd-472e-bee6-1d0d2701dd28
192026-10-22T19:57:00.505ZINFOdns-server (store): pruning trees for generations newer than 0
202026-10-22T19:57:00.521ZINFOdns-server (store): pruning trees for generations older than 1 keep = 3
212026-10-22T19:57:00.521ZINFOdns-server (store): updated generation new_generation = 1 req_id = 0644d019-1ddd-472e-bee6-1d0d2701dd28
222026-10-22T19:57:00.521ZINFOdns-server (http): request completed latency_us = 16744 local_addr = [fd00:1122:3344:3::1]:5353 method = PUT remote_addr = [fd00:1122:3344:3::2]:59356 req_id = 0644d019-1ddd-472e-bee6-1d0d2701dd28 response_code = 204 uri = /config
232026-10-22T19:57:11.449ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 42861, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:48645 req_id = fd50606c-a6b6-4442-98d0-4f33c2699641
242026-10-22T19:57:11.457ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61203, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:49477 req_id = 531ff51e-4c00-4dd9-a739-ea302dd52537
252026-10-22T19:57:11.463ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 36576, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:64242 req_id = fd7ee279-235c-4e30-bbd0-364ef8567a2b
262026-10-22T19:58:54.519ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57749, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:55241 req_id = 85cff86b-3c1e-4160-887f-f19fc9a25043
272026-10-22T19:58:56.497ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35111, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144."), ), original: Query { name: Name("oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 53, 98, 57, 99, 51, 56, 102, 51, 45, 54, 54, 98, 52, 45, 52, 101, 100, 101, 45, 57, 55, 51, 100, 45, 55, 101, 49, 57, 54, 49, 56, 51, 99, 49, 52, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::e]:57799 req_id = edaaeb8e-9042-4311-b50e-8f907977ab75
282026-10-22T19:58:56.503ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35111, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144."), ), original: Query { name: Name("oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 53, 98, 57, 99, 51, 56, 102, 51, 45, 54, 54, 98, 52, 45, 52, 101, 100, 101, 45, 57, 55, 51, 100, 45, 55, 101, 49, 57, 54, 49, 56, 51, 99, 49, 52, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::e]:45307 req_id = fc9a81af-3d81-474e-aef6-e7b687f55b81
292026-10-22T19:58:56.867ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 16914, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:63477 req_id = fba67436-76c9-48f3-9154-66c940bab0da
302026-10-22T19:58:57.581ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 33541, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:56575 req_id = b469486b-d7c5-473c-bc50-761c9bbe6c7a
312026-10-22T19:58:58.445ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38235, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:54454 req_id = 239e2f74-a5bd-4cb3-897e-3ac763cc271a
322026-10-22T19:59:01.168ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52961, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:53688 req_id = 02115ad8-21b6-4070-9275-fee74353ca25
332026-10-22T19:59:05.394ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 46275, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:53523 req_id = 6efea23c-9cf2-4045-8038-e1d5e4e7496c
342026-10-22T19:59:06.556ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52070, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::b]:30653 req_id = ffe21b8b-366f-4521-acc3-aca9e3b284f6
352026-10-22T19:59:06.576ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57831, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::c]:37073 req_id = 54593518-b55e-4fea-a164-1a89c136bf24
362026-10-22T19:59:06.841ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 34436, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::a]:16958 req_id = e04779a6-aea9-4253-abde-57f9a436b4e9
372026-10-22T19:59:12.780ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5239, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:35614 req_id = f0dae3cb-2c9d-49aa-bf39-3479c28ffa93
382026-10-22T19:59:12.786ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 1702, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:43338 req_id = 73875726-2ff4-4fde-ae26-e75a896e3873
392026-10-22T19:59:12.791ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 47924, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:50645 req_id = ea560f34-281f-42a1-ba86-e3fa858f0d3c
402026-10-22T19:59:14.828ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 62512, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:57043 req_id = 7100818d-030b-4188-97f2-ab0babb62b6c
412026-10-22T19:59:17.706ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 16336, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::c]:42118 req_id = 550f6a48-490c-48f4-a714-2a3bb47d55cb
422026-10-22T19:59:17.964ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::c]:53028
432026-10-22T19:59:17.994ZINFOdns-server (http): request completed latency_us = 15954 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:53028 req_id = db0ba8e6-2fdf-4aee-828b-08d13e05d8e8 response_code = 200 uri = /config
442026-10-22T19:59:20.373ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::c]:37977
452026-10-22T19:59:20.374ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = a4905c04-0c65-4fb5-9825-33f500105a7d
462026-10-22T19:59:20.374ZINFOdns-server (store): updated generation new_generation = 1 req_id = a4905c04-0c65-4fb5-9825-33f500105a7d
472026-10-22T19:59:20.374ZINFOdns-server (http): request completed latency_us = 305 local_addr = [fd00:1122:3344:3::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::c]:37977 req_id = a4905c04-0c65-4fb5-9825-33f500105a7d response_code = 204 uri = /config
482026-10-22T19:59:20.390ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14414, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::c]:3610 req_id = af9f3a40-b967-4d61-8bff-121a5fc1a2c7
492026-10-22T19:59:20.531ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::c]:62731
502026-10-22T19:59:20.532ZINFOdns-server (http): request completed latency_us = 261 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:62731 req_id = 16cd6f54-1203-4e32-85b5-673a214254ad response_code = 200 uri = /config
512026-10-22T19:59:23.142ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59365, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::c]:62228 req_id = 86e1821b-5efe-4f51-b868-b23705871f78
522026-10-22T19:59:23.360ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::c]:62942
532026-10-22T19:59:23.360ZINFOdns-server (http): request completed latency_us = 287 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:62942 req_id = 19ae6c59-253b-4da4-8acb-cd6b2e358da0 response_code = 200 uri = /config
542026-10-22T19:59:27.153ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 43102, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::b]:15629 req_id = c2242429-8573-4e3d-90b8-29342601a416
552026-10-22T19:59:27.236ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 627, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::a]:7581 req_id = f6d98ddd-b7b3-4211-82e4-64aa750cdb78
562026-10-22T19:59:27.282ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::b]:57314
572026-10-22T19:59:27.282ZINFOdns-server (http): request completed latency_us = 203 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:57314 req_id = c8af15a3-9c64-4394-99b4-97b1093b9c69 response_code = 200 uri = /config
582026-10-22T19:59:27.355ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::a]:42251
592026-10-22T19:59:27.355ZINFOdns-server (http): request completed latency_us = 421 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:42251 req_id = 3ac7deac-27de-4be9-8c46-793cfb924be7 response_code = 200 uri = /config
602026-10-22T19:59:32.320ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61990, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:53150 req_id = 304478bf-a1e9-453e-a5bf-6a8f9eee4b37
612026-10-22T19:59:42.322ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50315, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:51264 req_id = 6286688f-93b8-4fd0-b2a9-7d9c548b46e4
622026-10-22T19:59:52.323ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57216, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:54133 req_id = 5e184029-1dab-4b91-9bd2-25bacc8d6dce
632026-10-22T20:00:00.098ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45293, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::2]:61208 req_id = c12ca53e-e156-494b-85e8-b26e59975be3
642026-10-22T20:00:00.099ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45293, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::2]:52426 req_id = 57e05f6f-7c9a-45cf-a6ae-f12f318e2d2d
652026-10-22T20:00:00.493ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 16336, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_ntp_d542ae82-0649-4e92-a46b-351032921e8d."), ), original: Query { name: Name("oxz_ntp_d542ae82-0649-4e92-a46b-351032921e8d."), query_type: A, query_class: IN, }, }, ], original: [ 44, 111, 120, 122, 95, 110, 116, 112, 95, 100, 53, 52, 50, 97, 101, 56, 50, 45, 48, 54, 52, 57, 45, 52, 101, 57, 50, 45, 97, 52, 54, 98, 45, 51, 53, 49, 48, 51, 50, 57, 50, 49, 101, 56, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_ntp_d542ae82-0649-4e92-a46b-351032921e8d." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:44799 req_id = e3fb39fd-0d39-4849-88df-8db23901b211
662026-10-22T20:00:00.675ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 56621, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_855d2e42-8d09-4b2f-aa2f-d93908aead85."), ), original: Query { name: Name("oxz_cockroachdb_855d2e42-8d09-4b2f-aa2f-d93908aead85."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 56, 53, 53, 100, 50, 101, 52, 50, 45, 56, 100, 48, 57, 45, 52, 98, 50, 102, 45, 97, 97, 50, 102, 45, 100, 57, 51, 57, 48, 56, 97, 101, 97, 100, 56, 53, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_855d2e42-8d09-4b2f-aa2f-d93908aead85." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::3]:60327 req_id = 0631926b-dccd-4b28-a7af-6fafab5fbcec
672026-10-22T20:00:00.676ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 56621, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_855d2e42-8d09-4b2f-aa2f-d93908aead85."), ), original: Query { name: Name("oxz_cockroachdb_855d2e42-8d09-4b2f-aa2f-d93908aead85."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 56, 53, 53, 100, 50, 101, 52, 50, 45, 56, 100, 48, 57, 45, 52, 98, 50, 102, 45, 97, 97, 50, 102, 45, 100, 57, 51, 57, 48, 56, 97, 101, 97, 100, 56, 53, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_855d2e42-8d09-4b2f-aa2f-d93908aead85." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::3]:49485 req_id = 1698cbe2-dba3-4c79-b82f-1ad6a37cb5e3
682026-10-22T20:00:00.738ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 21807, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144."), ), original: Query { name: Name("oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 53, 98, 57, 99, 51, 56, 102, 51, 45, 54, 54, 98, 52, 45, 52, 101, 100, 101, 45, 57, 55, 51, 100, 45, 55, 101, 49, 57, 54, 49, 56, 51, 99, 49, 52, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::e]:41857 req_id = db26c76e-ae1b-4b05-a924-88df3322423b
692026-10-22T20:00:00.739ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 21807, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144."), ), original: Query { name: Name("oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 53, 98, 57, 99, 51, 56, 102, 51, 45, 54, 54, 98, 52, 45, 52, 101, 100, 101, 45, 57, 55, 51, 100, 45, 55, 101, 49, 57, 54, 49, 56, 51, 99, 49, 52, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_5b9c38f3-66b4-4ede-973d-7e196183c144." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::e]:49906 req_id = b369dd92-2004-45f4-9b1c-111ed87a5d53
702026-10-22T20:00:00.789ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19577, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_b961a81b-6cc6-450f-9712-42e72d1c30cc."), ), original: Query { name: Name("oxz_cockroachdb_b961a81b-6cc6-450f-9712-42e72d1c30cc."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 98, 57, 54, 49, 97, 56, 49, 98, 45, 54, 99, 99, 54, 45, 52, 53, 48, 102, 45, 57, 55, 49, 50, 45, 52, 50, 101, 55, 50, 100, 49, 99, 51, 48, 99, 99, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_b961a81b-6cc6-450f-9712-42e72d1c30cc." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::5]:43702 req_id = 708949c8-f478-4d7a-b07f-80e65486c2a9
712026-10-22T20:00:00.790ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19577, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_b961a81b-6cc6-450f-9712-42e72d1c30cc."), ), original: Query { name: Name("oxz_cockroachdb_b961a81b-6cc6-450f-9712-42e72d1c30cc."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 98, 57, 54, 49, 97, 56, 49, 98, 45, 54, 99, 99, 54, 45, 52, 53, 48, 102, 45, 57, 55, 49, 50, 45, 52, 50, 101, 55, 50, 100, 49, 99, 51, 48, 99, 99, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_b961a81b-6cc6-450f-9712-42e72d1c30cc." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::5]:48587 req_id = 650ecb60-4aff-4330-a7bf-f1ca37b93a3e
722026-10-22T20:00:00.807ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 17061, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_5657ccc7-0c36-48f0-b7d0-9f13a8327702."), ), original: Query { name: Name("oxz_cockroachdb_5657ccc7-0c36-48f0-b7d0-9f13a8327702."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 53, 54, 53, 55, 99, 99, 99, 55, 45, 48, 99, 51, 54, 45, 52, 56, 102, 48, 45, 98, 55, 100, 48, 45, 57, 102, 49, 51, 97, 56, 51, 50, 55, 55, 48, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_5657ccc7-0c36-48f0-b7d0-9f13a8327702." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::4]:47106 req_id = 16683a6f-3920-4dcb-937a-8adb088dc034
732026-10-22T20:00:00.808ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 17061, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_5657ccc7-0c36-48f0-b7d0-9f13a8327702."), ), original: Query { name: Name("oxz_cockroachdb_5657ccc7-0c36-48f0-b7d0-9f13a8327702."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 53, 54, 53, 55, 99, 99, 99, 55, 45, 48, 99, 51, 54, 45, 52, 56, 102, 48, 45, 98, 55, 100, 48, 45, 57, 102, 49, 51, 97, 56, 51, 50, 55, 55, 48, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_5657ccc7-0c36-48f0-b7d0-9f13a8327702." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::4]:60237 req_id = f3ad2305-5f8c-4774-a9a7-658bd9ecd7e4
742026-10-22T20:00:00.809ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38401, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_f78afd39-f97b-4168-964d-7da53fd37c2f."), ), original: Query { name: Name("oxz_cockroachdb_f78afd39-f97b-4168-964d-7da53fd37c2f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 102, 55, 56, 97, 102, 100, 51, 57, 45, 102, 57, 55, 98, 45, 52, 49, 54, 56, 45, 57, 54, 52, 100, 45, 55, 100, 97, 53, 51, 102, 100, 51, 55, 99, 50, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_f78afd39-f97b-4168-964d-7da53fd37c2f." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::6]:49256 req_id = 7a985dde-2148-4511-bcea-405068bf3b41
752026-10-22T20:00:00.809ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38401, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_f78afd39-f97b-4168-964d-7da53fd37c2f."), ), original: Query { name: Name("oxz_cockroachdb_f78afd39-f97b-4168-964d-7da53fd37c2f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 102, 55, 56, 97, 102, 100, 51, 57, 45, 102, 57, 55, 98, 45, 52, 49, 54, 56, 45, 57, 54, 52, 100, 45, 55, 100, 97, 53, 51, 102, 100, 51, 55, 99, 50, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_f78afd39-f97b-4168-964d-7da53fd37c2f." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::6]:61147 req_id = 9192f441-cecf-4ad6-ad8e-72be08cbdd6e
762026-10-22T20:00:00.825ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54545, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_a1a46669-9941-4f2b-abbd-db4471dc2a0f."), ), original: Query { name: Name("oxz_cockroachdb_a1a46669-9941-4f2b-abbd-db4471dc2a0f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 97, 49, 97, 52, 54, 54, 54, 57, 45, 57, 57, 52, 49, 45, 52, 102, 50, 98, 45, 97, 98, 98, 100, 45, 100, 98, 52, 52, 55, 49, 100, 99, 50, 97, 48, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_a1a46669-9941-4f2b-abbd-db4471dc2a0f." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::7]:59789 req_id = d8890458-fa1d-4b88-89c4-9e109eb5e1d5
772026-10-22T20:00:00.825ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54545, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_a1a46669-9941-4f2b-abbd-db4471dc2a0f."), ), original: Query { name: Name("oxz_cockroachdb_a1a46669-9941-4f2b-abbd-db4471dc2a0f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 97, 49, 97, 52, 54, 54, 54, 57, 45, 57, 57, 52, 49, 45, 52, 102, 50, 98, 45, 97, 98, 98, 100, 45, 100, 98, 52, 52, 55, 49, 100, 99, 50, 97, 48, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_a1a46669-9941-4f2b-abbd-db4471dc2a0f." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::7]:38820 req_id = c43b1fd1-e1bc-42f4-b7d7-2dbbe678a79b