Max level shown:
1 [ Oct 30 17:56:47 Enabled. ]
2 [ Oct 30 17:56:47 Rereading configuration. ]
3 [ Oct 30 17:56:48 Rereading configuration. ]
4 [ Oct 30 17:56:50 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dns-server/bin/dns-server --config-file /var/svc/manifest/site/internal_dns/config.toml --http-address [fd00:1122:3344:3::1]:5353 --dns-addresses [fd00:1122:3344:3::1]:53 &"). ]
5 [ Oct 30 17:56:50 Method "start" exited with status 0. ]
6 Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:3::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } }
7 note: configured to log to "/dev/stdout"
82026-10-30T17:56:50.245ZINFOdns-server: config config = Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:3::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } } dns_config = Config { bind_addresses: [[fd00:1122:3344:3::1]:53] }
92026-10-30T17:56:50.245ZINFOdns-server (store): opening sled database path = /data/dns
102026-10-30T17:56:50.267ZINFOdns-server (store): pruning trees for generations newer than 0
112026-10-30T17:56:50.268ZINFOdns-server (store): pruning trees for generations older than 0 keep = 3
122026-10-30T17:56:50.268ZINFOdns-server (dns): DNS server bound to address bind_address = [fd00:1122:3344:3::1]:53
132026-10-30T17:56:50.268ZINFOdns-server (http): listening local_addr = [fd00:1122:3344:3::1]:5353
142026-10-30T17:56:50.461ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:3::2]:63323
152026-10-30T17:56:50.462ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = b0619d82-5750-4621-a921-3fbe323484a9
162026-10-30T17:56:50.462ZINFOdns-server (store): pruning trees for generations newer than 0
172026-10-30T17:56:50.502ZINFOdns-server (store): pruning trees for generations older than 1 keep = 3
182026-10-30T17:56:50.502ZINFOdns-server (store): updated generation new_generation = 1 req_id = b0619d82-5750-4621-a921-3fbe323484a9
192026-10-30T17:56:50.502ZINFOdns-server (http): request completed latency_us = 40390 local_addr = [fd00:1122:3344:3::1]:5353 method = PUT remote_addr = [fd00:1122:3344:3::2]:63323 req_id = b0619d82-5750-4621-a921-3fbe323484a9 response_code = 204 uri = /config
202026-10-30T17:57:04.319ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3286, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:42121 req_id = 513630bf-b269-4f8f-aee4-7d2f2266739f
212026-10-30T17:57:04.324ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 10059, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:42162 req_id = 6ddb296c-3dce-48ea-bee2-526b9da65c87
222026-10-30T17:57:04.329ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59158, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:44670 req_id = 70ede688-6f12-4933-84d4-c380d7ca8db7
232026-10-30T17:57:32.477ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19323, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:56715 req_id = 1a7175b7-6638-4dec-b3c0-c0ff1cbba972
242026-10-30T17:57:32.482ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41894, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:37973 req_id = c6dfc423-855f-4432-a5e8-01d0aea29fa6
252026-10-30T17:57:32.487ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32125, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:35603 req_id = 35962fab-5132-45ec-b7fe-f380e4ae4cd5
262026-10-30T17:59:01.140ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 43742, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:52300 req_id = da309ef9-cd30-49e2-ba58-4c79e9324bfd
272026-10-30T17:59:03.103ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50776, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::e]:51839 req_id = f48d7143-ffa7-4b83-b128-d70d09cba9f8
282026-10-30T17:59:03.104ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50776, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::e]:38145 req_id = 57a7bad4-248c-491f-8ec9-1136ac4e2e08
292026-10-30T17:59:04.224ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 53221, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:52792 req_id = 261bc685-92d3-4cef-a8df-442b31765714
302026-10-30T17:59:04.776ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6068, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:63874 req_id = bc1009b5-6d49-495b-98f1-c38ae0ba3ba1
312026-10-30T17:59:05.619ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35231, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:56470 req_id = cb86d47f-e06d-4988-b12d-84f164da48df
322026-10-30T17:59:08.010ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18668, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:52291 req_id = 77de8838-99f8-413d-84fb-e1be26b89469
332026-10-30T17:59:10.949ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59074, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::a]:56261 req_id = 0749b5c0-c0d9-48bc-842c-ed7f0679b93e
342026-10-30T17:59:11.047ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 26231, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::c]:47306 req_id = b702c241-f3d7-493b-86b3-72cd4d2fe555
352026-10-30T17:59:11.119ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32240, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::b]:45783 req_id = 73aac4a0-107b-4a8b-bf58-d2b23942aec9
362026-10-30T17:59:12.215ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 30214, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:65501 req_id = 84bb2046-f05c-4f2d-b2e1-8cfb545c1b41
372026-10-30T17:59:13.147ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::b]:53948
382026-10-30T17:59:13.166ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::b]:35679
392026-10-30T17:59:13.166ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::a]:50812
402026-10-30T17:59:13.167ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::c]:35471
412026-10-30T17:59:13.174ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 7a346769-d988-4881-800a-61d88f880c0f
422026-10-30T17:59:13.174ZINFOdns-server (store): updated generation new_generation = 1 req_id = 7a346769-d988-4881-800a-61d88f880c0f
432026-10-30T17:59:13.175ZINFOdns-server (http): request completed latency_us = 14353 local_addr = [fd00:1122:3344:3::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::b]:53948 req_id = 7a346769-d988-4881-800a-61d88f880c0f response_code = 204 uri = /config
442026-10-30T17:59:13.175ZINFOdns-server (http): request completed latency_us = 7543 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:35471 req_id = 7f923515-e92b-40b6-b9af-398663d32968 response_code = 200 uri = /config
452026-10-30T17:59:13.175ZINFOdns-server (http): request completed latency_us = 7567 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:50812 req_id = 9cd7a9f2-4eca-448a-a94a-c832a7d57609 response_code = 200 uri = /config
462026-10-30T17:59:13.175ZINFOdns-server (http): request completed latency_us = 8600 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:35679 req_id = 4a86ca9c-fdee-4d63-ae36-58e63d37f916 response_code = 200 uri = /config
472026-10-30T17:59:13.527ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11684, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::c]:58162 req_id = 58a3902d-44bb-4c12-a265-1fc34ffc2c08
482026-10-30T17:59:13.653ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::c]:55438
492026-10-30T17:59:13.653ZINFOdns-server (http): request completed latency_us = 241 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:55438 req_id = 72ebc088-90e4-434a-84b1-73e056b65e7c response_code = 200 uri = /config
502026-10-30T17:59:14.884ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 60548, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::b]:6940 req_id = 813089f9-a58b-4e86-a33b-e5d744bb7390
512026-10-30T17:59:15.009ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::b]:32910
522026-10-30T17:59:15.010ZINFOdns-server (http): request completed latency_us = 234 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:32910 req_id = 8c116d6f-4c63-401c-8c6f-9c6fc79b25c9 response_code = 200 uri = /config
532026-10-30T17:59:15.534ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59289, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::b]:36101 req_id = 896d7f18-df74-4eda-85e1-f95af890d296
542026-10-30T17:59:15.774ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::b]:48458
552026-10-30T17:59:15.774ZINFOdns-server (http): request completed latency_us = 348 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:48458 req_id = 3a170579-f66b-4fd7-8fb0-64d588b1d9d1 response_code = 200 uri = /config
562026-10-30T17:59:19.425ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 9060, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:57148 req_id = b0f33ece-004f-4447-a5f8-04261a06f482
572026-10-30T17:59:21.592ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 60642, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::c]:12630 req_id = a3f88fc6-408c-47fc-8229-14b4503329ef
582026-10-30T17:59:21.593ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 2298, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::a]:2542 req_id = f2c666cc-dd8f-493a-af10-c88e78aadc8c
592026-10-30T17:59:21.714ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::a]:37388
602026-10-30T17:59:21.714ZINFOdns-server (http): request completed latency_us = 210 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:37388 req_id = dbbbdfdd-7991-471c-8a4f-79e682d94e54 response_code = 200 uri = /config
612026-10-30T17:59:21.764ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:3::1]:5353 remote_addr = [fd00:1122:3344:101::c]:45899
622026-10-30T17:59:21.764ZINFOdns-server (http): request completed latency_us = 195 local_addr = [fd00:1122:3344:3::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:45899 req_id = f1ee8c7c-8d95-407d-a5d5-b3f68b4b6ad8 response_code = 200 uri = /config
632026-10-30T17:59:29.426ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 8404, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:51192 req_id = 7e0377e0-1259-43dc-8a25-6565de9c3900
642026-10-30T17:59:39.427ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 47474, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:55188 req_id = 995a3665-fc62-49cd-8c42-decb38ea02e6
652026-10-30T17:59:49.430ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 47978, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:56115 req_id = 626d94cb-953d-452e-a8dd-b5d028752b19
662026-10-30T17:59:59.431ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 44979, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::d]:54453 req_id = 62b828fb-8c79-41b9-9ec9-e330c70f29f4
672026-10-30T18:00:00.333ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6720, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), ), original: Query { name: Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 50, 48, 57, 101, 102, 49, 100, 50, 45, 52, 98, 97, 51, 45, 52, 52, 51, 100, 45, 56, 57, 50, 99, 45, 53, 54, 57, 100, 57, 48, 102, 99, 51, 50, 48, 56, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::3]:34744 req_id = 00701390-fe41-4f3c-b056-a7fb976e1189
682026-10-30T18:00:00.334ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6720, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), ), original: Query { name: Name("oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 50, 48, 57, 101, 102, 49, 100, 50, 45, 52, 98, 97, 51, 45, 52, 52, 51, 100, 45, 56, 57, 50, 99, 45, 53, 54, 57, 100, 57, 48, 102, 99, 51, 50, 48, 56, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_209ef1d2-4ba3-443d-892c-569d90fc3208." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::3]:54488 req_id = cdec1788-0d3c-4143-8a4f-b0eb3ff012ac
692026-10-30T18:00:00.423ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32519, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::e]:48842 req_id = b579868f-bc72-4874-a6cf-a1a77956f6bd
702026-10-30T18:00:00.423ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32519, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), ), original: Query { name: Name("oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 52, 51, 48, 50, 101, 54, 100, 98, 45, 56, 51, 52, 57, 45, 52, 97, 53, 54, 45, 57, 53, 101, 100, 45, 52, 98, 54, 50, 54, 49, 51, 100, 101, 49, 99, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_4302e6db-8349-4a56-95ed-4b62613de1c4." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::e]:43648 req_id = 84356ea1-cfb8-40ff-9270-0dc4ee49200e
712026-10-30T18:00:00.445ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14065, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), ), original: Query { name: Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 48, 57, 52, 56, 50, 98, 53, 48, 45, 100, 54, 99, 51, 45, 52, 48, 101, 57, 45, 56, 54, 49, 56, 45, 97, 100, 49, 56, 97, 97, 101, 50, 98, 51, 98, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::6]:36765 req_id = db0cbfff-c2d5-4ad2-8f85-ef20caeb9681
722026-10-30T18:00:00.445ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14065, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), ), original: Query { name: Name("oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 48, 57, 52, 56, 50, 98, 53, 48, 45, 100, 54, 99, 51, 45, 52, 48, 101, 57, 45, 56, 54, 49, 56, 45, 97, 100, 49, 56, 97, 97, 101, 50, 98, 51, 98, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_09482b50-d6c3-40e9-8618-ad18aae2b3bd." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::6]:60204 req_id = e40d1c20-c4d3-47ae-825c-956929070f45
732026-10-30T18:00:00.508ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3458, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), ), original: Query { name: Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 98, 57, 101, 54, 101, 98, 53, 102, 45, 52, 49, 99, 55, 45, 52, 99, 102, 99, 45, 98, 97, 97, 53, 45, 52, 49, 102, 48, 53, 99, 56, 97, 97, 98, 98, 97, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::5]:44875 req_id = 9152ea84-8d1b-437a-ad85-b79c24695860
742026-10-30T18:00:00.508ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3458, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), ), original: Query { name: Name("oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 98, 57, 101, 54, 101, 98, 53, 102, 45, 52, 49, 99, 55, 45, 52, 99, 102, 99, 45, 98, 97, 97, 53, 45, 52, 49, 102, 48, 53, 99, 56, 97, 97, 98, 98, 97, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_b9e6eb5f-41c7-4cfc-baa5-41f05c8aabba." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::5]:60418 req_id = 52e3956d-8842-48a1-b3a9-231466df7bf0
752026-10-30T18:00:00.521ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37661, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), ), original: Query { name: Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 52, 49, 51, 49, 101, 53, 49, 45, 98, 52, 54, 99, 45, 52, 53, 101, 99, 45, 57, 51, 54, 52, 45, 54, 49, 100, 100, 55, 51, 98, 56, 48, 97, 98, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::4]:50289 req_id = 637b0d5f-4edd-4b41-affe-ed2a0d33a94f
762026-10-30T18:00:00.522ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37661, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), ), original: Query { name: Name("oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 57, 52, 49, 51, 49, 101, 53, 49, 45, 98, 52, 54, 99, 45, 52, 53, 101, 99, 45, 57, 51, 54, 52, 45, 54, 49, 100, 100, 55, 51, 98, 56, 48, 97, 98, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_94131e51-b46c-45ec-9364-61dd73b80ab0." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::4]:51744 req_id = f55edd9a-59e0-4f69-b89e-c77b06425815
772026-10-30T18:00:00.668ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22392, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), ), original: Query { name: Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 56, 97, 97, 52, 51, 97, 50, 56, 45, 52, 57, 56, 97, 45, 52, 48, 50, 52, 45, 56, 99, 99, 54, 45, 100, 100, 99, 99, 49, 99, 56, 53, 100, 56, 57, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::7]:46204 req_id = e956f2d4-8c4f-470f-b74c-670b7873e196
782026-10-30T18:00:00.669ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22392, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), ), original: Query { name: Name("oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 56, 97, 97, 52, 51, 97, 50, 56, 45, 52, 57, 56, 97, 45, 52, 48, 50, 52, 45, 56, 99, 99, 54, 45, 100, 100, 99, 99, 49, 99, 56, 53, 100, 56, 57, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_8aa43a28-498a-4024-8cc6-ddcc1c85d89f." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::7]:51745 req_id = e525b9ce-3b37-4ec1-9ea7-7fdcd370ae62
792026-10-30T18:00:00.694ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18528, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_ntp_e3812aa9-6b3c-447b-bc17-94eb9ff785ff."), ), original: Query { name: Name("oxz_ntp_e3812aa9-6b3c-447b-bc17-94eb9ff785ff."), query_type: A, query_class: IN, }, }, ], original: [ 44, 111, 120, 122, 95, 110, 116, 112, 95, 101, 51, 56, 49, 50, 97, 97, 57, 45, 54, 98, 51, 99, 45, 52, 52, 55, 98, 45, 98, 99, 49, 55, 45, 57, 52, 101, 98, 57, 102, 102, 55, 56, 53, 102, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_ntp_e3812aa9-6b3c-447b-bc17-94eb9ff785ff." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::1b]:65459 req_id = 8f35d294-35be-4676-bebd-87679fd9f8f3
802026-10-30T18:00:01.136ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32623, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::2]:60344 req_id = 674e3dd3-191b-4ca2-bbb3-06334690b9d7
812026-10-30T18:00:01.136ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32623, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." bind_address = [fd00:1122:3344:3::1]:53 peer_addr = [fd00:1122:3344:101::2]:57006 req_id = 42a697bd-c18f-4a7c-a099-8fa7318d5db6