Max level shown:
1 [ Oct 30 17:56:47 Enabled. ]
2 [ Oct 30 17:56:47 Rereading configuration. ]
3 [ Oct 30 17:56:48 Rereading configuration. ]
4 [ Oct 30 17:56:50 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dns-server/bin/dns-server --config-file /var/svc/manifest/site/internal_dns/config.toml --http-address [fd00:1122:3344:1::1]:5353 --dns-addresses [fd00:1122:3344:1::1]:53 &"). ]
5 [ Oct 30 17:56:50 Method "start" exited with status 0. ]
6 Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:1::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } }
7 note: configured to log to "/dev/stdout"
82026-10-30T17:56:50.265ZINFOdns-server: config config = Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:1::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } } dns_config = Config { bind_addresses: [[fd00:1122:3344:1::1]:53] }
92026-10-30T17:56:50.266ZINFOdns-server (store): opening sled database path = /data/dns
102026-10-30T17:56:50.285ZINFOdns-server (store): pruning trees for generations newer than 0
112026-10-30T17:56:50.285ZINFOdns-server (store): pruning trees for generations older than 0 keep = 3
122026-10-30T17:56:50.285ZINFOdns-server (dns): DNS server bound to address bind_address = [fd00:1122:3344:1::1]:53
132026-10-30T17:56:50.285ZINFOdns-server (http): listening local_addr = [fd00:1122:3344:1::1]:5353
142026-10-30T17:56:50.414ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:1::2]:47736
152026-10-30T17:56:50.414ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 8b773505-4da3-4221-b083-1c8b0f780f91
162026-10-30T17:56:50.414ZINFOdns-server (store): pruning trees for generations newer than 0
172026-10-30T17:56:50.460ZINFOdns-server (store): pruning trees for generations older than 1 keep = 3
182026-10-30T17:56:50.460ZINFOdns-server (store): updated generation new_generation = 1 req_id = 8b773505-4da3-4221-b083-1c8b0f780f91
192026-10-30T17:56:50.460ZINFOdns-server (http): request completed latency_us = 46421 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:1::2]:47736 req_id = 8b773505-4da3-4221-b083-1c8b0f780f91 response_code = 204 uri = /config
202026-10-30T17:57:04.318ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3286, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:50297 req_id = 2c9be836-9d99-4950-a4f5-85b11b6b9d6a
212026-10-30T17:57:04.323ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 10059, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:60722 req_id = 350613de-6e46-47f5-a593-d57f69113001
222026-10-30T17:57:04.328ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59158, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:45421 req_id = 76abb1fe-63dc-4150-9028-4cd16b710bf3
232026-10-30T17:57:32.477ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19323, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:36959 req_id = dbb99574-b0e3-4977-b0bf-21f8f8bc751f
242026-10-30T17:57:32.482ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41894, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:64878 req_id = b1365d56-c2e9-45ca-b3ba-e08c8ec52cbe
252026-10-30T17:57:32.487ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32125, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:36879 req_id = d1fd4643-9c0d-498f-8853-64051815d758
262026-10-30T17:59:01.148ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38407, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:54813 req_id = 9decbe39-a566-4ccd-9752-b20fb8a914ad
272026-10-30T17:59:03.097ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50776, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::e]:36625 req_id = 3481b349-6ad2-47a8-98e2-fb005ffbd6a2
282026-10-30T17:59:03.103ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50776, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::e]:39843 req_id = 0b59e83f-9ae3-4949-8b03-ab2116ef3776
292026-10-30T17:59:04.224ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 33610, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:54835 req_id = e323c704-8353-42ea-8975-41bca51ce1fa
302026-10-30T17:59:04.776ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5863, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:54572 req_id = 112e2c40-4c32-4d6a-b992-eba99dbf158a
312026-10-30T17:59:05.619ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 9648, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:56560 req_id = 717dd5df-135a-499b-81ad-1100111abfb4
322026-10-30T17:59:08.010ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11537, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:59354 req_id = 0533f450-bf2b-4879-92a2-7a828a65aa0b
332026-10-30T17:59:10.949ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11973, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::a]:31707 req_id = a465416a-2348-4475-b547-286d8c61a8e5
342026-10-30T17:59:11.047ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 40689, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::c]:50618 req_id = 79939e95-3907-4b8d-9f0c-0cbee10a588a
352026-10-30T17:59:11.119ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5670, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::b]:21777 req_id = ad6695ad-defe-4f63-9639-0193c7b0bc71
362026-10-30T17:59:12.215ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54300, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:57173 req_id = 23ff6b45-76e4-460b-94b6-d5b6753bc0d1
372026-10-30T17:59:13.144ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:59412
382026-10-30T17:59:13.166ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:59033
392026-10-30T17:59:13.166ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:34656
402026-10-30T17:59:13.166ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:65378
412026-10-30T17:59:13.174ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 718c4459-91ca-4333-a0b8-798ffe3adaa3
422026-10-30T17:59:13.174ZINFOdns-server (store): updated generation new_generation = 1 req_id = 718c4459-91ca-4333-a0b8-798ffe3adaa3
432026-10-30T17:59:13.176ZINFOdns-server (http): request completed latency_us = 15305 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::b]:59412 req_id = 718c4459-91ca-4333-a0b8-798ffe3adaa3 response_code = 204 uri = /config
442026-10-30T17:59:13.177ZINFOdns-server (http): request completed latency_us = 8210 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:34656 req_id = ef15e231-6528-4875-95e7-29175eaeb5bb response_code = 200 uri = /config
452026-10-30T17:59:13.177ZINFOdns-server (http): request completed latency_us = 8277 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:59033 req_id = cc98499b-99d1-4497-9e20-988cde9e962d response_code = 200 uri = /config
462026-10-30T17:59:13.177ZINFOdns-server (http): request completed latency_us = 6166 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:65378 req_id = a255cd8c-085e-4048-8bdb-a6589402f09e response_code = 200 uri = /config
472026-10-30T17:59:13.527ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41749, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::c]:35717 req_id = ad322373-b396-4e9f-a306-b28ba7006fc0
482026-10-30T17:59:13.653ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:61876
492026-10-30T17:59:13.653ZINFOdns-server (http): request completed latency_us = 244 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:61876 req_id = 50c2b80b-2ad0-41d3-ae6c-91861629301a response_code = 200 uri = /config
502026-10-30T17:59:14.884ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 64047, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::b]:64786 req_id = 2d43351d-a1f4-435c-a5cf-194aa67b7e84
512026-10-30T17:59:15.009ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:39695
522026-10-30T17:59:15.009ZINFOdns-server (http): request completed latency_us = 239 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:39695 req_id = ff18f98b-92bb-4b55-bcae-4d024b34fee6 response_code = 200 uri = /config
532026-10-30T17:59:15.534ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50882, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::b]:1682 req_id = 913179b4-d687-40d4-9560-6643b65bfaeb
542026-10-30T17:59:15.774ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:54872
552026-10-30T17:59:15.774ZINFOdns-server (http): request completed latency_us = 246 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:54872 req_id = a021818a-179d-480e-9cff-b3c67318ef99 response_code = 200 uri = /config
562026-10-30T17:59:19.425ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 39232, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:63204 req_id = 924af241-94a0-4cd5-96a2-6c27d84e148d
572026-10-30T17:59:21.592ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54827, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::a]:61312 req_id = 22a1cdfd-8ba1-4a73-a127-898136964598
582026-10-30T17:59:21.592ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 2222, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::c]:61855 req_id = 4f07ffda-a171-41f8-bdf2-565f77d121a9
592026-10-30T17:59:21.714ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:50324
602026-10-30T17:59:21.714ZINFOdns-server (http): request completed latency_us = 287 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:50324 req_id = 94a8589f-e67a-47bc-9151-9a792f922ef2 response_code = 200 uri = /config
612026-10-30T17:59:21.764ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:33649
622026-10-30T17:59:21.764ZINFOdns-server (http): request completed latency_us = 192 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:33649 req_id = 0676c2e0-3600-4973-9ecf-67504491e406 response_code = 200 uri = /config
632026-10-30T17:59:29.426ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 34569, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:63923 req_id = e58f7b27-9c48-4c68-bb77-ac263ad84ff2
642026-10-30T17:59:39.428ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59318, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:59299 req_id = 76916015-e98f-4e48-98a6-9a3bbfa83647
652026-10-30T17:59:49.430ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61315, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:59321 req_id = 54fa5747-0d04-4bf8-908f-08cf86032000
662026-10-30T17:59:59.431ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 47375, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:57296 req_id = aae49502-6f1a-4139-801e-976dc979f238
672026-10-30T18:00:00.333ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6720, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), ), original: Query { name: Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 50, 48, 57, 101, 102, 49, 100, 50, 45, 52, 98, 97, 51, 45, 52, 52, 51, 100, 45, 56, 57, 50, 99, 45, 53, 54, 57, 100, 57, 48, 102, 99, 51, 50, 48, 56, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::3]:58745 req_id = c918cd60-d89c-44b5-acc4-6e51bd90406c
682026-10-30T18:00:00.333ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6720, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), ), original: Query { name: Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 50, 48, 57, 101, 102, 49, 100, 50, 45, 52, 98, 97, 51, 45, 52, 52, 51, 100, 45, 56, 57, 50, 99, 45, 53, 54, 57, 100, 57, 48, 102, 99, 51, 50, 48, 56, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::3]:35610 req_id = 6612018e-746a-4ddb-83ac-210828adc39d
692026-10-30T18:00:00.422ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32519, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::e]:44914 req_id = bb3be18a-2899-4b1f-a138-2206ba70e9af
702026-10-30T18:00:00.423ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32519, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::e]:57599 req_id = 95cd85c0-c1e1-46f2-8ae9-9e496e000703
712026-10-30T18:00:00.444ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14065, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), ), original: Query { name: Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 48, 57, 52, 56, 50, 98, 53, 48, 45, 100, 54, 99, 51, 45, 52, 48, 101, 57, 45, 56, 54, 49, 56, 45, 97, 100, 49, 56, 97, 97, 101, 50, 98, 51, 98, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::6]:46575 req_id = 91043985-b7c3-497c-81fa-8b6b7e638770
722026-10-30T18:00:00.445ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14065, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), ), original: Query { name: Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 48, 57, 52, 56, 50, 98, 53, 48, 45, 100, 54, 99, 51, 45, 52, 48, 101, 57, 45, 56, 54, 49, 56, 45, 97, 100, 49, 56, 97, 97, 101, 50, 98, 51, 98, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::6]:33890 req_id = d6ac9113-f808-4d84-9b49-7631ce87d0f5
732026-10-30T18:00:00.507ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3458, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), ), original: Query { name: Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 98, 57, 101, 54, 101, 98, 53, 102, 45, 52, 49, 99, 55, 45, 52, 99, 102, 99, 45, 98, 97, 97, 53, 45, 52, 49, 102, 48, 53, 99, 56, 97, 97, 98, 98, 97, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::5]:41939 req_id = 301d11be-b987-4a03-8c55-4b4438329a61
742026-10-30T18:00:00.508ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3458, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), ), original: Query { name: Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 98, 57, 101, 54, 101, 98, 53, 102, 45, 52, 49, 99, 55, 45, 52, 99, 102, 99, 45, 98, 97, 97, 53, 45, 52, 49, 102, 48, 53, 99, 56, 97, 97, 98, 98, 97, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::5]:61172 req_id = 7a0bf8cc-6472-4e0a-9b0a-3172e41e95c1
752026-10-30T18:00:00.521ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37661, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), ), original: Query { name: Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 52, 49, 51, 49, 101, 53, 49, 45, 98, 52, 54, 99, 45, 52, 53, 101, 99, 45, 57, 51, 54, 52, 45, 54, 49, 100, 100, 55, 51, 98, 56, 48, 97, 98, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::4]:62643 req_id = 2b2c16bb-fe7c-41cc-b218-a48beda35a24
762026-10-30T18:00:00.521ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37661, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), ), original: Query { name: Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 52, 49, 51, 49, 101, 53, 49, 45, 98, 52, 54, 99, 45, 52, 53, 101, 99, 45, 57, 51, 54, 52, 45, 54, 49, 100, 100, 55, 51, 98, 56, 48, 97, 98, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::4]:45533 req_id = 10e32641-39cc-4c7b-9ab5-471ba7213e91
772026-10-30T18:00:00.667ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22392, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), ), original: Query { name: Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 56, 97, 97, 52, 51, 97, 50, 56, 45, 52, 57, 56, 97, 45, 52, 48, 50, 52, 45, 56, 99, 99, 54, 45, 100, 100, 99, 99, 49, 99, 56, 53, 100, 56, 57, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::7]:51594 req_id = a7f71a9f-9293-46a3-8c0b-1283ca3ea3ae
782026-10-30T18:00:00.668ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22392, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), ), original: Query { name: Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 56, 97, 97, 52, 51, 97, 50, 56, 45, 52, 57, 56, 97, 45, 52, 48, 50, 52, 45, 56, 99, 99, 54, 45, 100, 100, 99, 99, 49, 99, 56, 53, 100, 56, 57, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::7]:41450 req_id = 877f195e-c387-4d44-8ba5-6a8cf0f2face
792026-10-30T18:00:00.693ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18528, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_ntp_e3812aa9-6b3c-447b-bc17-94eb9ff785ff."), ), original: Query { name: Name("oxz_ntp_e3812aa9-6b3c-447b-bc17-94eb9ff785ff."), query_type: A, query_class: IN, }, }, ], original: [ 44, 111, 120, 122, 95, 110, 116, 112, 95, 101, 51, 56, 49, 50, 97, 97, 57, 45, 54, 98, 51, 99, 45, 52, 52, 55, 98, 45, 98, 99, 49, 55, 45, 57, 52, 101, 98, 57, 102, 102, 55, 56, 53, 102, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_ntp_e3812aa9-6b3c-447b-bc17-94eb9ff785ff." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:65461 req_id = 8428719d-6994-47b7-b0dd-841e1d528ec1
802026-10-30T18:00:01.135ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32623, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::2]:47496 req_id = cccf0855-9352-41f3-949d-c426a7669259
812026-10-30T18:00:01.136ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32623, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::2]:58036 req_id = 5e1b5aff-c410-4158-94d3-9f52b2bb35ca