Max level shown:
1 [ Oct 30 17:56:47 Enabled. ]
2 [ Oct 30 17:56:47 Rereading configuration. ]
3 [ Oct 30 17:56:48 Rereading configuration. ]
4 [ Oct 30 17:56:50 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dns-server/bin/dns-server --config-file /var/svc/manifest/site/internal_dns/config.toml --http-address [fd00:1122:3344:2::1]:5353 --dns-addresses [fd00:1122:3344:2::1]:53 &"). ]
5 [ Oct 30 17:56:50 Method "start" exited with status 0. ]
6 Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:2::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } }
7 note: configured to log to "/dev/stdout"
82026-10-30T17:56:50.348ZINFOdns-server: config config = Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:2::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } } dns_config = Config { bind_addresses: [[fd00:1122:3344:2::1]:53] }
92026-10-30T17:56:50.348ZINFOdns-server (store): opening sled database path = /data/dns
102026-10-30T17:56:50.374ZINFOdns-server (store): pruning trees for generations newer than 0
112026-10-30T17:56:50.374ZINFOdns-server (store): pruning trees for generations older than 0 keep = 3
122026-10-30T17:56:50.374ZINFOdns-server (dns): DNS server bound to address bind_address = [fd00:1122:3344:2::1]:53
132026-10-30T17:56:50.374ZINFOdns-server (http): listening local_addr = [fd00:1122:3344:2::1]:5353
142026-10-30T17:56:50.503ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:2::2]:39708
152026-10-30T17:56:50.504ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 6d4f76ee-fc25-49b2-97b8-fbe1fa183084
162026-10-30T17:56:50.504ZINFOdns-server (store): pruning trees for generations newer than 0
172026-10-30T17:56:50.543ZINFOdns-server (store): pruning trees for generations older than 1 keep = 3
182026-10-30T17:56:50.543ZINFOdns-server (store): updated generation new_generation = 1 req_id = 6d4f76ee-fc25-49b2-97b8-fbe1fa183084
192026-10-30T17:56:50.543ZINFOdns-server (http): request completed latency_us = 39716 local_addr = [fd00:1122:3344:2::1]:5353 method = PUT remote_addr = [fd00:1122:3344:2::2]:39708 req_id = 6d4f76ee-fc25-49b2-97b8-fbe1fa183084 response_code = 204 uri = /config
202026-10-30T17:57:04.318ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3286, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::1b]:42298 req_id = 151c378b-75f0-4037-b934-103c5353536f
212026-10-30T17:57:04.323ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 10059, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::1b]:45172 req_id = 73619b78-46eb-4e9a-b78b-5bbe58c9a72d
222026-10-30T17:57:04.329ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59158, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::1b]:42386 req_id = aeae7500-c249-44d8-8b37-a7ac44d74b17
232026-10-30T17:57:32.477ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19323, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::1b]:41247 req_id = 6c119e33-3d6e-449f-ac80-c090c53b740b
242026-10-30T17:57:32.481ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41894, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::1b]:40436 req_id = be4ac59a-1504-4ea7-b210-4b5f5ae0caac
252026-10-30T17:57:32.487ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32125, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::1b]:39127 req_id = f4d5a7a4-7f01-4cd6-b4c3-cbb3325c3f80
262026-10-30T17:59:01.142ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 40635, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:54800 req_id = 31dfb850-266f-414f-a6e0-2f322768d458
272026-10-30T17:59:03.100ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50776, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::e]:64373 req_id = 68921ca8-336c-4a72-9388-7d69102ae565
282026-10-30T17:59:03.103ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50776, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::e]:37197 req_id = 75788721-0a18-4d15-b8fb-977997811f75
292026-10-30T17:59:04.224ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 39616, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:55116 req_id = 7c69c807-5dd3-4775-9bf1-a018b0c5b6d2
302026-10-30T17:59:04.776ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 53560, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:56858 req_id = b416deda-cd16-4b1c-8d93-fcc733ea2cee
312026-10-30T17:59:05.619ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 29301, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:54832 req_id = bccafcbd-13b6-4e00-9fcd-883472449c1f
322026-10-30T17:59:08.010ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38863, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:57045 req_id = 37c30568-6ced-4f8d-932e-82db8cf785a1
332026-10-30T17:59:10.949ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 56413, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::a]:62893 req_id = ed3ad09d-89b0-436d-9387-cc6c7e253d6d
342026-10-30T17:59:11.047ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 34364, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::c]:15203 req_id = d719e3a3-4a4b-49b1-b712-55d258a10b8e
352026-10-30T17:59:11.119ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 24556, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::b]:3311 req_id = ad007f55-5c1b-41e2-8b00-e6c121e789c3
362026-10-30T17:59:12.215ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 63863, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:56513 req_id = ded57c8b-6319-4946-9f6c-902519a6d118
372026-10-30T17:59:13.141ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::b]:46806
382026-10-30T17:59:13.166ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::b]:44438
392026-10-30T17:59:13.167ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::a]:43756
402026-10-30T17:59:13.167ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::c]:52606
412026-10-30T17:59:13.174ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 04f4feba-b9ef-4c83-addf-0072e47e0e10
422026-10-30T17:59:13.174ZINFOdns-server (store): updated generation new_generation = 1 req_id = 04f4feba-b9ef-4c83-addf-0072e47e0e10
432026-10-30T17:59:13.174ZINFOdns-server (http): request completed latency_us = 17915 local_addr = [fd00:1122:3344:2::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::b]:46806 req_id = 04f4feba-b9ef-4c83-addf-0072e47e0e10 response_code = 204 uri = /config
442026-10-30T17:59:13.174ZINFOdns-server (http): request completed latency_us = 6063 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:43756 req_id = 298cdaed-a5af-44e0-8e9d-9cd5f5ac5077 response_code = 200 uri = /config
452026-10-30T17:59:13.174ZINFOdns-server (http): request completed latency_us = 6953 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:52606 req_id = a407e8d2-d9a0-4ed5-8820-677726baeffc response_code = 200 uri = /config
462026-10-30T17:59:13.174ZINFOdns-server (http): request completed latency_us = 6993 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:44438 req_id = ef5ebabb-1a6d-4f2f-8eb7-58b138c9f683 response_code = 200 uri = /config
472026-10-30T17:59:13.527ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54020, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::c]:9104 req_id = 650faeae-2ffa-44e5-9fa7-0abf2e3f6f4e
482026-10-30T17:59:13.653ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::c]:37933
492026-10-30T17:59:13.653ZINFOdns-server (http): request completed latency_us = 258 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:37933 req_id = 7062276b-88cd-46f2-9d6e-20922f49b3f2 response_code = 200 uri = /config
502026-10-30T17:59:14.884ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18726, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::b]:14927 req_id = f07dccb7-55b2-48a8-b9b9-6bc2e618887e
512026-10-30T17:59:15.009ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::b]:38526
522026-10-30T17:59:15.010ZINFOdns-server (http): request completed latency_us = 263 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:38526 req_id = 25c53943-6fe6-4789-abe8-1bbdd7c0310c response_code = 200 uri = /config
532026-10-30T17:59:15.534ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55892, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::b]:61913 req_id = 31b9c9c9-fc1e-4051-a0a2-bcb6512652b3
542026-10-30T17:59:15.774ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::b]:54497
552026-10-30T17:59:15.774ZINFOdns-server (http): request completed latency_us = 219 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:54497 req_id = 36fe5c8d-d006-4081-a252-450a511f08d9 response_code = 200 uri = /config
562026-10-30T17:59:19.425ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 10245, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:59981 req_id = 1a70cffa-d6e1-4012-b976-94e0bbb80d09
572026-10-30T17:59:21.592ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5649, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::c]:28326 req_id = 35ce1164-988e-499b-a879-9898f99fc12e
582026-10-30T17:59:21.592ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 62232, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::a]:5311 req_id = ac459bcf-dd39-4f27-987d-84ab9b26086a
592026-10-30T17:59:21.714ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::a]:60834
602026-10-30T17:59:21.714ZINFOdns-server (http): request completed latency_us = 292 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:60834 req_id = 988d943b-6a82-4bca-bbcc-4949c8bd4cbb response_code = 200 uri = /config
612026-10-30T17:59:21.764ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:2::1]:5353 remote_addr = [fd00:1122:3344:101::c]:41769
622026-10-30T17:59:21.764ZINFOdns-server (http): request completed latency_us = 244 local_addr = [fd00:1122:3344:2::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:41769 req_id = e7a6768f-d3f0-4767-ae7b-9f0a098ae998 response_code = 200 uri = /config
632026-10-30T17:59:29.426ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 49133, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:50140 req_id = 13d91464-507c-4332-b32c-313065606b27
642026-10-30T17:59:39.427ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59005, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:61836 req_id = 226ceb1a-b62f-43e7-b536-4cd2abaa8492
652026-10-30T17:59:49.430ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19388, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:49760 req_id = 85f87ad5-a70f-40a7-b211-3e0f3fbcf1e9
662026-10-30T17:59:59.431ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18430, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::d]:56740 req_id = 948ea15e-a5c1-48e6-bfd6-797626180b61
672026-10-30T18:00:00.333ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6720, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), ), original: Query { name: Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 50, 48, 57, 101, 102, 49, 100, 50, 45, 52, 98, 97, 51, 45, 52, 52, 51, 100, 45, 56, 57, 50, 99, 45, 53, 54, 57, 100, 57, 48, 102, 99, 51, 50, 48, 56, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::3]:37500 req_id = f193a066-0176-4264-bd21-0d6419da8dbc
682026-10-30T18:00:00.333ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6720, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), ), original: Query { name: Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 50, 48, 57, 101, 102, 49, 100, 50, 45, 52, 98, 97, 51, 45, 52, 52, 51, 100, 45, 56, 57, 50, 99, 45, 53, 54, 57, 100, 57, 48, 102, 99, 51, 50, 48, 56, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::3]:63532 req_id = 48713173-c78e-464b-856d-be68f15124ec
692026-10-30T18:00:00.422ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32519, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::e]:50654 req_id = 260b9e84-849e-423a-bd93-836b630efd71
702026-10-30T18:00:00.423ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32519, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::e]:51594 req_id = bc098fe5-459c-4fb0-b57d-ddaaf7fde471
712026-10-30T18:00:00.445ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14065, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), ), original: Query { name: Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 48, 57, 52, 56, 50, 98, 53, 48, 45, 100, 54, 99, 51, 45, 52, 48, 101, 57, 45, 56, 54, 49, 56, 45, 97, 100, 49, 56, 97, 97, 101, 50, 98, 51, 98, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::6]:45593 req_id = 21152051-3a4b-47fb-a88e-7e9b9e711104
722026-10-30T18:00:00.445ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14065, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), ), original: Query { name: Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 48, 57, 52, 56, 50, 98, 53, 48, 45, 100, 54, 99, 51, 45, 52, 48, 101, 57, 45, 56, 54, 49, 56, 45, 97, 100, 49, 56, 97, 97, 101, 50, 98, 51, 98, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::6]:34611 req_id = 08efb3ce-5951-4515-965d-5e0724873973
732026-10-30T18:00:00.507ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3458, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), ), original: Query { name: Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 98, 57, 101, 54, 101, 98, 53, 102, 45, 52, 49, 99, 55, 45, 52, 99, 102, 99, 45, 98, 97, 97, 53, 45, 52, 49, 102, 48, 53, 99, 56, 97, 97, 98, 98, 97, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::5]:60221 req_id = 5b7dd636-e56e-478e-afc6-d0e5173493b4
742026-10-30T18:00:00.508ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3458, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), ), original: Query { name: Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 98, 57, 101, 54, 101, 98, 53, 102, 45, 52, 49, 99, 55, 45, 52, 99, 102, 99, 45, 98, 97, 97, 53, 45, 52, 49, 102, 48, 53, 99, 56, 97, 97, 98, 98, 97, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::5]:37822 req_id = c34852f2-a6ac-406a-8b3f-520617131d85
752026-10-30T18:00:00.521ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37661, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), ), original: Query { name: Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 52, 49, 51, 49, 101, 53, 49, 45, 98, 52, 54, 99, 45, 52, 53, 101, 99, 45, 57, 51, 54, 52, 45, 54, 49, 100, 100, 55, 51, 98, 56, 48, 97, 98, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::4]:57813 req_id = 34684c3e-dcd4-466a-aa0f-53da6e50a182
762026-10-30T18:00:00.521ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37661, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), ), original: Query { name: Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 52, 49, 51, 49, 101, 53, 49, 45, 98, 52, 54, 99, 45, 52, 53, 101, 99, 45, 57, 51, 54, 52, 45, 54, 49, 100, 100, 55, 51, 98, 56, 48, 97, 98, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::4]:40472 req_id = 4fab4259-5ebd-4c00-b8e6-020104d9e777
772026-10-30T18:00:00.668ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22392, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), ), original: Query { name: Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 56, 97, 97, 52, 51, 97, 50, 56, 45, 52, 57, 56, 97, 45, 52, 48, 50, 52, 45, 56, 99, 99, 54, 45, 100, 100, 99, 99, 49, 99, 56, 53, 100, 56, 57, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::7]:38982 req_id = baddf4e2-1149-400d-bf53-2f5450fca2cf
782026-10-30T18:00:00.668ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22392, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), ), original: Query { name: Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 56, 97, 97, 52, 51, 97, 50, 56, 45, 52, 57, 56, 97, 45, 52, 48, 50, 52, 45, 56, 99, 99, 54, 45, 100, 100, 99, 99, 49, 99, 56, 53, 100, 56, 57, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::7]:43741 req_id = 23e20533-5c9d-4a46-8174-665ecc51f3f0
792026-10-30T18:00:00.694ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18528, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_ntp_e3812aa9-6b3c-447b-bc17-94eb9ff785ff."), ), original: Query { name: Name("oxz_ntp_e3812aa9-6b3c-447b-bc17-94eb9ff785ff."), query_type: A, query_class: IN, }, }, ], original: [ 44, 111, 120, 122, 95, 110, 116, 112, 95, 101, 51, 56, 49, 50, 97, 97, 57, 45, 54, 98, 51, 99, 45, 52, 52, 55, 98, 45, 98, 99, 49, 55, 45, 57, 52, 101, 98, 57, 102, 102, 55, 56, 53, 102, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_ntp_e3812aa9-6b3c-447b-bc17-94eb9ff785ff." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::1b]:51804 req_id = 601fd67b-0667-43b2-915a-ec4395ba0579
802026-10-30T18:00:01.135ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32623, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::2]:43311 req_id = 9f8a8465-a273-436b-af6b-95e2b42fa6ea
812026-10-30T18:00:01.136ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32623, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." bind_address = [fd00:1122:3344:2::1]:53 peer_addr = [fd00:1122:3344:101::2]:65299 req_id = 88407814-46f2-4857-9670-2a3b68dab120