Max level shown:
1 [ Jul 29 21:25:54 Enabled. ]
2 [ Jul 29 21:25:54 Rereading configuration. ]
3 [ Jul 29 21:25:54 Rereading configuration. ]
4 [ Jul 29 21:25:56 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dns-server/bin/dns-server --config-file /var/svc/manifest/site/internal_dns/config.toml --http-address [fd00:1122:3344:1::1]:5353 --dns-address [fd00:1122:3344:1::1]:53 &"). ]
5 [ Jul 29 21:25:56 Method "start" exited with status 0. ]
6 Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:1::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } }
7 note: configured to log to "/dev/stdout"
82026-07-29T21:25:56.690ZINFOdns-server: config config = Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:1::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } } dns_config = Config { bind_address: [fd00:1122:3344:1::1]:53 }
92026-07-29T21:25:56.690ZINFOdns-server (store): opening sled database path = /data/dns
102026-07-29T21:25:56.711ZINFOdns-server (store): pruning trees for generations newer than 0
112026-07-29T21:25:56.711ZINFOdns-server (store): pruning trees for generations older than 0 keep = 3
122026-07-29T21:25:56.711ZINFOdns-server (dns): DNS server bound to address local_address = [fd00:1122:3344:1::1]:53
132026-07-29T21:25:56.712ZINFOdns-server (http): listening local_addr = [fd00:1122:3344:1::1]:5353
142026-07-29T21:25:59.050ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59014, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:1::2]:11453 req_id = 526ba74f-7b88-44a7-9f4b-39eb45c41ce2
152026-07-29T21:26:00.056ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 63388, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:1::2]:13986 req_id = 447bcba4-8e48-4b70-b50a-4eac3cfd922f
162026-07-29T21:26:00.672ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 40181, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:51581 req_id = f2f51c14-44b0-40a4-845c-b43c6d5be13c
172026-07-29T21:26:00.801ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52394, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:49362 req_id = 75d31d6c-95fe-4c00-a47a-56387f5575a3
182026-07-29T21:26:00.961ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37369, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:7097 req_id = 0728c852-b170-47a5-8f1f-1040a3f3c4ad
192026-07-29T21:26:00.974ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 27952, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:42125 req_id = cce2e9c8-fe4b-49fc-9a6b-1817827936c0
202026-07-29T21:26:01.046ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61360, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:1::2]:53874 req_id = 7922580f-e602-4057-ae0e-38653451f8a1
212026-07-29T21:26:01.295ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57864, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:45624 req_id = dba46f05-5aaa-4f69-a80a-47fad869fe52
222026-07-29T21:26:01.529ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 63484, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:4467 req_id = 4708657d-bb68-40c8-9a99-020a1e950c32
232026-07-29T21:26:02.263ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 63093, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:47005 req_id = a45d9e07-677f-49ca-9fea-b809baaeae6c
242026-07-29T21:26:02.776ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 10224, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:24532 req_id = 958286e5-5250-45ee-b8d4-4adb08f029e8
252026-07-29T21:26:03.046ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35497, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:1::2]:24827 req_id = a2289665-bb41-4463-90ca-27cd3ccc3290
262026-07-29T21:26:03.943ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 49165, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:46583 req_id = 6d9c8291-8cdf-4f29-92aa-35c8432ec6ce
272026-07-29T21:26:04.382ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:1::2]:58839
282026-07-29T21:26:04.382ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 59189d19-d238-4cdc-b167-8e4f09113646
292026-07-29T21:26:04.382ZINFOdns-server (store): pruning trees for generations newer than 0
302026-07-29T21:26:04.396ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 26031, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_nexus._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 6, 95, 110, 101, 120, 117, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "_nexus._tcp.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:11932 req_id = 64100cb8-8886-4f8f-972c-83c6de54b72f
312026-07-29T21:26:04.397ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14081, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("8a6d7931-75ff-4c25-aab4-0b7211fc2ced.host.control-plane.oxide.internal."), ), original: Query { name: Name("8a6d7931-75ff-4c25-aab4-0b7211fc2ced.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN, }, }, ], original: [ 36, 56, 97, 54, 100, 55, 57, 51, 49, 45, 55, 53, 102, 102, 45, 52, 99, 50, 53, 45, 97, 97, 98, 52, 45, 48, 98, 55, 50, 49, 49, 102, 99, 50, 99, 101, 100, 4, 104, 111, 115, 116, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "8a6d7931-75ff-4c25-aab4-0b7211fc2ced.host.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:8441 req_id = c6b1d662-4b8a-49a9-b389-6b8e185732c6
322026-07-29T21:26:04.397ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 25681, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("7495615a-bf4a-4adb-8f22-76b6ec0eeaf0.host.control-plane.oxide.internal."), ), original: Query { name: Name("7495615a-bf4a-4adb-8f22-76b6ec0eeaf0.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN, }, }, ], original: [ 36, 55, 52, 57, 53, 54, 49, 53, 97, 45, 98, 102, 52, 97, 45, 52, 97, 100, 98, 45, 56, 102, 50, 50, 45, 55, 54, 98, 54, 101, 99, 48, 101, 101, 97, 102, 48, 4, 104, 111, 115, 116, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "7495615a-bf4a-4adb-8f22-76b6ec0eeaf0.host.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:5386 req_id = 72d7c180-425b-4c35-b0a0-970cfaf27882
332026-07-29T21:26:04.398ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 62248, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("c4d5473a-4104-4cc4-a8e3-5014d5a958dc.host.control-plane.oxide.internal."), ), original: Query { name: Name("c4d5473a-4104-4cc4-a8e3-5014d5a958dc.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN, }, }, ], original: [ 36, 99, 52, 100, 53, 52, 55, 51, 97, 45, 52, 49, 48, 52, 45, 52, 99, 99, 52, 45, 97, 56, 101, 51, 45, 53, 48, 49, 52, 100, 53, 97, 57, 53, 56, 100, 99, 4, 104, 111, 115, 116, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } SERVFAIL: server is not authoritative for name: "c4d5473a-4104-4cc4-a8e3-5014d5a958dc.host.control-plane.oxide.internal." peer_addr = [fd00:1122:3344:101::2]:53231 req_id = 8a0fab08-067b-4f28-834b-5c17a09da20c
342026-07-29T21:26:04.421ZINFOdns-server (store): pruning trees for generations older than 1 keep = 3
352026-07-29T21:26:04.421ZINFOdns-server (store): updated generation new_generation = 1 req_id = 59189d19-d238-4cdc-b167-8e4f09113646
362026-07-29T21:26:04.421ZINFOdns-server (http): request completed latency_us = 38770 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:1::2]:58839 req_id = 59189d19-d238-4cdc-b167-8e4f09113646 response_code = 204 uri = /config
372026-07-29T21:26:19.246ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 16083, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:35518 req_id = b9da431a-a5c2-4b73-8f36-d70dd2d3f3ae
382026-07-29T21:26:19.251ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38384, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:64705 req_id = ed8b2da7-9a9c-4f99-b5c6-0704d511e12e
392026-07-29T21:26:19.256ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 15981, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:60301 req_id = fd703ce1-9ea9-4f57-b13f-bfc882b51d5a
402026-07-29T21:28:19.597ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 64527, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:52474 req_id = 0d58c2e4-b973-4fdf-8865-5b7016910a9c
412026-07-29T21:28:19.644ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 60595, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:34572 req_id = 40894d41-4bc2-4d72-9a63-1c69b3dc0c02
422026-07-29T21:28:19.648ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 48171, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:57045 req_id = c7ce75ac-f36e-45f9-aa38-1ce54930ca74
432026-07-29T21:28:26.135ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11203, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:56064 req_id = 9f7fc600-fa8b-4f3c-b7fd-d5f0453690c4
442026-07-29T21:28:36.141ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 13497, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:53016 req_id = 8bec9c7a-ba5d-48f6-bd01-ca43cd6edb53
452026-07-29T21:28:36.385ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 7477, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::b]:11466 req_id = b233df05-51c9-41fa-996e-dea38ab6a2a2
462026-07-29T21:28:37.331ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57794, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:59184 req_id = 42fb1c54-3e8a-4263-b6cc-7c8eef2938a2
472026-07-29T21:28:37.387ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 12804, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:26254 req_id = 4d7495d4-c508-4b87-a715-93af320a42bb
482026-07-29T21:28:42.341ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 24578, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2."), ), original: Query { name: Name("oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 98, 97, 100, 49, 50, 57, 50, 56, 45, 99, 49, 57, 53, 45, 52, 49, 55, 57, 45, 98, 52, 53, 50, 45, 54, 51, 50, 99, 98, 100, 102, 100, 100, 102, 102, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2." peer_addr = [fd00:1122:3344:101::e]:60582 req_id = f3638a96-84e3-4a07-bf62-e267f9a0da2e
492026-07-29T21:28:42.342ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 24578, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2."), ), original: Query { name: Name("oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 98, 97, 100, 49, 50, 57, 50, 56, 45, 99, 49, 57, 53, 45, 52, 49, 55, 57, 45, 98, 52, 53, 50, 45, 54, 51, 50, 99, 98, 100, 102, 100, 100, 102, 102, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2." peer_addr = [fd00:1122:3344:101::e]:41257 req_id = 50b90eac-05c9-43c4-903b-c70745c68ca7
502026-07-29T21:28:46.143ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 43481, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:65263 req_id = 93d97602-997d-49cf-9a1c-c675bb583125
512026-07-29T21:28:56.145ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35504, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:51436 req_id = d47b6c83-54d6-4807-8c99-a41d492ee990
522026-07-29T21:29:06.147ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19664, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:59202 req_id = f5016760-93d2-4096-a918-90ba95ccd05d
532026-07-29T21:29:16.149ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11984, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:57740 req_id = 4cf4b75f-2986-4b2c-84c3-dbe7c50c19b5
542026-07-29T21:29:20.634ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41347, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:34483 req_id = dc590d80-f346-42d3-90b7-4b6d840df0b9
552026-07-29T21:29:20.923ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:63547
562026-07-29T21:29:20.983ZINFOdns-server (http): request completed latency_us = 31812 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:63547 req_id = 3ff42860-0f2d-439a-b3b2-c4f4ad4eaaa4 response_code = 200 uri = /config
572026-07-29T21:29:26.152ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 16171, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:64769 req_id = 45b7fb07-87cf-45eb-bb1c-93aaf198195e
582026-07-29T21:29:29.264ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:65011
592026-07-29T21:29:29.267ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 09e03c0a-0b0d-461c-8f3c-93ab61d01345
602026-07-29T21:29:29.267ZINFOdns-server (store): updated generation new_generation = 1 req_id = 09e03c0a-0b0d-461c-8f3c-93ab61d01345
612026-07-29T21:29:29.267ZINFOdns-server (http): request completed latency_us = 2427 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::c]:65011 req_id = 09e03c0a-0b0d-461c-8f3c-93ab61d01345 response_code = 204 uri = /config
622026-07-29T21:29:29.581ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 65023, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:43803 req_id = f4a5cd20-f19d-48ff-a31a-213e3d761fbd
632026-07-29T21:29:29.699ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:34460
642026-07-29T21:29:29.699ZINFOdns-server (http): request completed latency_us = 225 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:34460 req_id = 7258da63-25eb-47a7-8295-5149eeddf58e response_code = 200 uri = /config
652026-07-29T21:29:30.321ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 2696, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:26597 req_id = c276ebdf-5255-42ad-8619-c65c1e1f9157
662026-07-29T21:29:30.426ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:45115
672026-07-29T21:29:30.426ZINFOdns-server (http): request completed latency_us = 222 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:45115 req_id = df4ace97-f89d-4039-a1e0-11801e0a659f response_code = 200 uri = /config
682026-07-29T21:29:36.154ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 48937, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:57129 req_id = ece3d402-cf88-4312-a71e-7708aa97fd21
692026-07-29T21:29:36.326ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:54273
702026-07-29T21:29:36.328ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 11e435da-0d48-41cc-8b36-48cf4f7b5dfa
712026-07-29T21:29:36.328ZINFOdns-server (store): updated generation new_generation = 1 req_id = 11e435da-0d48-41cc-8b36-48cf4f7b5dfa
722026-07-29T21:29:36.328ZINFOdns-server (http): request completed latency_us = 325 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::b]:54273 req_id = 11e435da-0d48-41cc-8b36-48cf4f7b5dfa response_code = 204 uri = /config
732026-07-29T21:29:37.078ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:36825
742026-07-29T21:29:37.080ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 32e67203-1130-413d-b1ef-41ad3aace3f7
752026-07-29T21:29:37.080ZINFOdns-server (store): updated generation new_generation = 1 req_id = 32e67203-1130-413d-b1ef-41ad3aace3f7
762026-07-29T21:29:37.080ZINFOdns-server (http): request completed latency_us = 257 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::c]:36825 req_id = 32e67203-1130-413d-b1ef-41ad3aace3f7 response_code = 204 uri = /config
772026-07-29T21:29:37.313ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:55089
782026-07-29T21:29:37.314ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = a8163b6a-4678-479c-9459-b4f8a04505c2
792026-07-29T21:29:37.314ZINFOdns-server (store): updated generation new_generation = 1 req_id = a8163b6a-4678-479c-9459-b4f8a04505c2
802026-07-29T21:29:37.315ZINFOdns-server (http): request completed latency_us = 375 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::a]:55089 req_id = a8163b6a-4678-479c-9459-b4f8a04505c2 response_code = 204 uri = /config
812026-07-29T21:29:37.532ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14470, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::b]:18200 req_id = 28901a14-d7b1-4ef2-a8b2-c8b6cf678835
822026-07-29T21:29:37.673ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:49052
832026-07-29T21:29:37.674ZINFOdns-server (http): request completed latency_us = 191 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:49052 req_id = 83938d7e-367c-4b62-991e-3eb41aeff3c9 response_code = 200 uri = /config
842026-07-29T21:29:38.118ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11122, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:19226 req_id = 49180e0d-7c2b-4e90-af65-acc381330fc2
852026-07-29T21:29:38.217ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:56636
862026-07-29T21:29:38.217ZINFOdns-server (http): request completed latency_us = 187 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:56636 req_id = b0dcc4c0-2887-47c7-b87f-94b3968f9276 response_code = 200 uri = /config
872026-07-29T21:29:38.697ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 63, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:2418 req_id = f13d9435-a322-49e7-8aab-777d2faae1c0
882026-07-29T21:29:38.795ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:51287
892026-07-29T21:29:38.796ZINFOdns-server (http): request completed latency_us = 200 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:51287 req_id = 0158b557-f31e-461d-8e06-abc65d3dae20 response_code = 200 uri = /config
902026-07-29T21:29:46.156ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45471, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:50969 req_id = 7978a0d3-abf9-4082-9a3b-56af51ea4b5d
912026-07-29T21:29:56.158ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38497, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:53863 req_id = 1e185f59-c37b-42ad-89f2-253a7267801d
922026-07-29T21:30:00.395ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61083, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_ntp_7ac1c757-c32e-4c72-9e08-0ff461730975."), ), original: Query { name: Name("oxz_ntp_7ac1c757-c32e-4c72-9e08-0ff461730975."), query_type: A, query_class: IN, }, }, ], original: [ 44, 111, 120, 122, 95, 110, 116, 112, 95, 55, 97, 99, 49, 99, 55, 53, 55, 45, 99, 51, 50, 101, 45, 52, 99, 55, 50, 45, 57, 101, 48, 56, 45, 48, 102, 102, 52, 54, 49, 55, 51, 48, 57, 55, 53, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_ntp_7ac1c757-c32e-4c72-9e08-0ff461730975." peer_addr = [fd00:1122:3344:101::1b]:56914 req_id = d3a025ff-d1d5-40ed-8d03-5a178f71e359
932026-07-29T21:30:00.647ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 28360, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2."), ), original: Query { name: Name("oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 98, 97, 100, 49, 50, 57, 50, 56, 45, 99, 49, 57, 53, 45, 52, 49, 55, 57, 45, 98, 52, 53, 50, 45, 54, 51, 50, 99, 98, 100, 102, 100, 100, 102, 102, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2." peer_addr = [fd00:1122:3344:101::e]:48545 req_id = e431621a-7a4b-4f50-9372-654808683994
942026-07-29T21:30:00.648ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 28360, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2."), ), original: Query { name: Name("oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 98, 97, 100, 49, 50, 57, 50, 56, 45, 99, 49, 57, 53, 45, 52, 49, 55, 57, 45, 98, 52, 53, 50, 45, 54, 51, 50, 99, 98, 100, 102, 100, 100, 102, 102, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_bad12928-c195-4179-b452-632cbdfddff2." peer_addr = [fd00:1122:3344:101::e]:45862 req_id = 08b6b57e-98c5-4415-8b8f-a047ba2b901d
952026-07-29T21:30:00.721ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 34439, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_3cd1cde4-7350-43ae-8529-99a419e4b300."), ), original: Query { name: Name("oxz_cockroachdb_3cd1cde4-7350-43ae-8529-99a419e4b300."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 51, 99, 100, 49, 99, 100, 101, 52, 45, 55, 51, 53, 48, 45, 52, 51, 97, 101, 45, 56, 53, 50, 57, 45, 57, 57, 97, 52, 49, 57, 101, 52, 98, 51, 48, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_3cd1cde4-7350-43ae-8529-99a419e4b300." peer_addr = [fd00:1122:3344:101::3]:37078 req_id = e1b97aff-583c-48ea-819a-223f476fa464
962026-07-29T21:30:00.722ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 34439, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_3cd1cde4-7350-43ae-8529-99a419e4b300."), ), original: Query { name: Name("oxz_cockroachdb_3cd1cde4-7350-43ae-8529-99a419e4b300."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 51, 99, 100, 49, 99, 100, 101, 52, 45, 55, 51, 53, 48, 45, 52, 51, 97, 101, 45, 56, 53, 50, 57, 45, 57, 57, 97, 52, 49, 57, 101, 52, 98, 51, 48, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_3cd1cde4-7350-43ae-8529-99a419e4b300." peer_addr = [fd00:1122:3344:101::3]:33462 req_id = 97adfc21-ead5-47bf-8b82-ee134015113e
972026-07-29T21:30:00.829ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38976, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_9d158613-13ae-40cd-a24b-d9385750fd00."), ), original: Query { name: Name("oxz_cockroachdb_9d158613-13ae-40cd-a24b-d9385750fd00."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 100, 49, 53, 56, 54, 49, 51, 45, 49, 51, 97, 101, 45, 52, 48, 99, 100, 45, 97, 50, 52, 98, 45, 100, 57, 51, 56, 53, 55, 53, 48, 102, 100, 48, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_9d158613-13ae-40cd-a24b-d9385750fd00." peer_addr = [fd00:1122:3344:101::6]:35040 req_id = b26ce316-04c2-475b-a28f-aa8d34c6268a
982026-07-29T21:30:00.830ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38976, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_9d158613-13ae-40cd-a24b-d9385750fd00."), ), original: Query { name: Name("oxz_cockroachdb_9d158613-13ae-40cd-a24b-d9385750fd00."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 100, 49, 53, 56, 54, 49, 51, 45, 49, 51, 97, 101, 45, 52, 48, 99, 100, 45, 97, 50, 52, 98, 45, 100, 57, 51, 56, 53, 55, 53, 48, 102, 100, 48, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_9d158613-13ae-40cd-a24b-d9385750fd00." peer_addr = [fd00:1122:3344:101::6]:48419 req_id = 8c36dbaf-7feb-4769-9b71-0ea8ef9f30a2
992026-07-29T21:30:00.869ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 17156, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." peer_addr = [fd00:1122:3344:101::2]:45035 req_id = da19778d-af49-4d02-b4ae-9800ebe9c3c5
1002026-07-29T21:30:00.869ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 17156, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." peer_addr = [fd00:1122:3344:101::2]:44051 req_id = 97dddbca-d0ee-414c-8629-53682ad8afc5
1012026-07-29T21:30:00.871ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37615, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_4af32ada-399d-4f75-8161-f8256ad5b451."), ), original: Query { name: Name("oxz_cockroachdb_4af32ada-399d-4f75-8161-f8256ad5b451."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 52, 97, 102, 51, 50, 97, 100, 97, 45, 51, 57, 57, 100, 45, 52, 102, 55, 53, 45, 56, 49, 54, 49, 45, 102, 56, 50, 53, 54, 97, 100, 53, 98, 52, 53, 49, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_4af32ada-399d-4f75-8161-f8256ad5b451." peer_addr = [fd00:1122:3344:101::4]:45164 req_id = f6b91810-e4ef-440e-8280-d506e6914380
1022026-07-29T21:30:00.871ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37615, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_4af32ada-399d-4f75-8161-f8256ad5b451."), ), original: Query { name: Name("oxz_cockroachdb_4af32ada-399d-4f75-8161-f8256ad5b451."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 52, 97, 102, 51, 50, 97, 100, 97, 45, 51, 57, 57, 100, 45, 52, 102, 55, 53, 45, 56, 49, 54, 49, 45, 102, 56, 50, 53, 54, 97, 100, 53, 98, 52, 53, 49, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_4af32ada-399d-4f75-8161-f8256ad5b451." peer_addr = [fd00:1122:3344:101::4]:39323 req_id = ac5ea44a-4583-4b85-bb76-ed9dbddd8625
1032026-07-29T21:30:00.918ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 33854, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_0df1112b-db48-482b-8423-f0290b6a7b24."), ), original: Query { name: Name("oxz_cockroachdb_0df1112b-db48-482b-8423-f0290b6a7b24."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 48, 100, 102, 49, 49, 49, 50, 98, 45, 100, 98, 52, 56, 45, 52, 56, 50, 98, 45, 56, 52, 50, 51, 45, 102, 48, 50, 57, 48, 98, 54, 97, 55, 98, 50, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_0df1112b-db48-482b-8423-f0290b6a7b24." peer_addr = [fd00:1122:3344:101::7]:54197 req_id = 2b0322ba-bf26-4047-881e-8a72ef688446
1042026-07-29T21:30:00.919ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 33854, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_0df1112b-db48-482b-8423-f0290b6a7b24."), ), original: Query { name: Name("oxz_cockroachdb_0df1112b-db48-482b-8423-f0290b6a7b24."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 48, 100, 102, 49, 49, 49, 50, 98, 45, 100, 98, 52, 56, 45, 52, 56, 50, 98, 45, 56, 52, 50, 51, 45, 102, 48, 50, 57, 48, 98, 54, 97, 55, 98, 50, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_0df1112b-db48-482b-8423-f0290b6a7b24." peer_addr = [fd00:1122:3344:101::7]:59457 req_id = 99953e91-d9bf-44c9-b1db-cbc8da2024c2
1052026-07-29T21:30:00.940ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3735, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_fcf27b2b-c431-4d1e-9531-7bc3320e6c1c."), ), original: Query { name: Name("oxz_cockroachdb_fcf27b2b-c431-4d1e-9531-7bc3320e6c1c."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 102, 99, 102, 50, 55, 98, 50, 98, 45, 99, 52, 51, 49, 45, 52, 100, 49, 101, 45, 57, 53, 51, 49, 45, 55, 98, 99, 51, 51, 50, 48, 101, 54, 99, 49, 99, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_fcf27b2b-c431-4d1e-9531-7bc3320e6c1c." peer_addr = [fd00:1122:3344:101::5]:51493 req_id = 3eaba808-0714-4573-8197-ec537f0a5dc3
1062026-07-29T21:30:00.940ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3735, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_fcf27b2b-c431-4d1e-9531-7bc3320e6c1c."), ), original: Query { name: Name("oxz_cockroachdb_fcf27b2b-c431-4d1e-9531-7bc3320e6c1c."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 102, 99, 102, 50, 55, 98, 50, 98, 45, 99, 52, 51, 49, 45, 52, 100, 49, 101, 45, 57, 53, 51, 49, 45, 55, 98, 99, 51, 51, 50, 48, 101, 54, 99, 49, 99, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_fcf27b2b-c431-4d1e-9531-7bc3320e6c1c." peer_addr = [fd00:1122:3344:101::5]:54966 req_id = baba600c-6b83-4f15-9f6b-33ab6e220740