Max level shown:
1 [ Sep 15 04:11:20 Enabled. ]
2 [ Sep 15 04:11:20 Rereading configuration. ]
3 [ Sep 15 04:11:21 Rereading configuration. ]
4 [ Sep 15 04:11:22 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dns-server/bin/dns-server --config-file /var/svc/manifest/site/internal_dns/config.toml --http-address [fd00:1122:3344:1::1]:5353 --dns-addresses [fd00:1122:3344:1::1]:53 &"). ]
5 [ Sep 15 04:11:22 Method "start" exited with status 0. ]
6 Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:1::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } }
7 note: configured to log to "/dev/stdout"
82026-09-15T04:11:22.409ZINFOdns-server: config config = Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:1::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } } dns_config = Config { bind_addresses: [[fd00:1122:3344:1::1]:53] }
92026-09-15T04:11:22.409ZINFOdns-server (store): opening sled database path = /data/dns
102026-09-15T04:11:22.431ZINFOdns-server (store): pruning trees for generations newer than 0
112026-09-15T04:11:22.431ZINFOdns-server (store): pruning trees for generations older than 0 keep = 3
122026-09-15T04:11:22.431ZINFOdns-server (dns): DNS server bound to address bind_address = [fd00:1122:3344:1::1]:53
132026-09-15T04:11:22.431ZINFOdns-server (http): listening local_addr = [fd00:1122:3344:1::1]:5353
142026-09-15T04:11:23.674ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:1::2]:55546
152026-09-15T04:11:23.675ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = f6b81886-d6a5-45b6-9fc4-cd507b46ec1f
162026-09-15T04:11:23.675ZINFOdns-server (store): pruning trees for generations newer than 0
172026-09-15T04:11:23.691ZINFOdns-server (store): pruning trees for generations older than 1 keep = 3
182026-09-15T04:11:23.691ZINFOdns-server (store): updated generation new_generation = 1 req_id = f6b81886-d6a5-45b6-9fc4-cd507b46ec1f
192026-09-15T04:11:23.691ZINFOdns-server (http): request completed latency_us = 16678 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:1::2]:55546 req_id = f6b81886-d6a5-45b6-9fc4-cd507b46ec1f response_code = 204 uri = /config
202026-09-15T04:11:35.868ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 2508, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:40248 req_id = ec392d0a-7e75-450e-80cc-40d9b9dcb733
212026-09-15T04:11:35.873ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55886, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:63553 req_id = d1ba8fed-56b9-48d8-a73f-8486f9903659
222026-09-15T04:11:35.879ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 7082, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:45804 req_id = 4f758db1-ba5c-472a-bd5b-b51cba919801
232026-09-15T04:12:42.097ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 1867, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:51403 req_id = 86e2d0dc-1002-454b-9880-721663eaab66
242026-09-15T04:12:47.556ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 30962, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2."), ), original: Query { name: Name("oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 49, 48, 101, 50, 52, 102, 102, 48, 45, 57, 97, 99, 98, 45, 52, 101, 53, 49, 45, 97, 97, 101, 48, 45, 51, 51, 101, 99, 101, 53, 100, 49, 50, 97, 99, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::e]:38309 req_id = 710143ae-8e79-4926-9c94-224847c147de
252026-09-15T04:12:47.563ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 30962, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2."), ), original: Query { name: Name("oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 49, 48, 101, 50, 52, 102, 102, 48, 45, 57, 97, 99, 98, 45, 52, 101, 53, 49, 45, 97, 97, 101, 48, 45, 51, 51, 101, 99, 101, 53, 100, 49, 50, 97, 99, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::e]:51849 req_id = 79679fb4-5296-4d22-b03f-e2660c0aed04
262026-09-15T04:12:52.084ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 4432, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:54879 req_id = 36a38c62-c4c4-4630-9b8b-2447083ca8ab
272026-09-15T04:12:54.494ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 30457, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::c]:29258 req_id = 4acb836d-3ece-4af8-9bb0-962ebe99d820
282026-09-15T04:12:54.554ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45641, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::a]:27220 req_id = 5aefa68d-652c-4383-bdc8-3771897da35d
292026-09-15T04:12:54.692ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 12806, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::b]:7030 req_id = ab09fe3a-23b5-4edb-9948-d37e04fa34a2
302026-09-15T04:13:02.086ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 27207, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:52971 req_id = 4312eaed-eac9-4d41-9433-b35c63fb1e82
312026-09-15T04:13:12.087ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 13565, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:53860 req_id = e6e376bc-4d27-42b7-a3e4-d94f61944f6f
322026-09-15T04:13:22.090ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41392, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:51930 req_id = 89e58100-0101-434f-bff8-55796d48868e
332026-09-15T04:13:32.091ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 30738, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:53198 req_id = 03286cd0-4b92-4f08-b7c8-ec6d858a9a07
342026-09-15T04:13:42.093ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 997, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:61817 req_id = 8f0cf777-1c5a-4ef7-9c05-037c72a59f85
352026-09-15T04:13:42.417ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 44444, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::a]:1700 req_id = 5129c4f2-51e0-4e34-a343-7fb52330dccd
362026-09-15T04:13:42.668ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:37627
372026-09-15T04:13:42.712ZINFOdns-server (http): request completed latency_us = 26858 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:37627 req_id = 9da48002-bd3d-4137-a096-d3caac872d52 response_code = 200 uri = /config
382026-09-15T04:13:46.228ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:48099
392026-09-15T04:13:46.231ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 252fa543-ae30-4e68-a5c0-ddb3c8afb458
402026-09-15T04:13:46.231ZINFOdns-server (store): updated generation new_generation = 1 req_id = 252fa543-ae30-4e68-a5c0-ddb3c8afb458
412026-09-15T04:13:46.231ZINFOdns-server (http): request completed latency_us = 2328 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::a]:48099 req_id = 252fa543-ae30-4e68-a5c0-ddb3c8afb458 response_code = 204 uri = /config
422026-09-15T04:13:46.445ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 12690, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::a]:64931 req_id = 5a9f3393-4188-4c7a-acab-4fe3274fe5c9
432026-09-15T04:13:46.551ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:39075
442026-09-15T04:13:46.551ZINFOdns-server (http): request completed latency_us = 253 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:39075 req_id = aecdb63a-df23-4ab4-8644-042e66925d3f response_code = 200 uri = /config
452026-09-15T04:13:46.967ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 9797, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::a]:46833 req_id = 62e44e2e-b31a-4e6a-8320-33ae44065184
462026-09-15T04:13:47.068ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:63780
472026-09-15T04:13:47.069ZINFOdns-server (http): request completed latency_us = 385 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:63780 req_id = e5579003-802e-42b3-b880-ebbbc6294904 response_code = 200 uri = /config
482026-09-15T04:13:52.095ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 56494, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:63865 req_id = d5b2becf-4f56-4906-812f-f581f0bd5503
492026-09-15T04:13:54.194ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:58290
502026-09-15T04:13:54.200ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = fa154a3e-aa00-4320-9c47-365011119126
512026-09-15T04:13:54.201ZINFOdns-server (store): updated generation new_generation = 1 req_id = fa154a3e-aa00-4320-9c47-365011119126
522026-09-15T04:13:54.201ZINFOdns-server (http): request completed latency_us = 281 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::a]:58290 req_id = fa154a3e-aa00-4320-9c47-365011119126 response_code = 204 uri = /config
532026-09-15T04:13:54.517ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:50652
542026-09-15T04:13:54.519ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = e54e53da-e71a-48f9-815c-d57359532d51
552026-09-15T04:13:54.519ZINFOdns-server (store): updated generation new_generation = 1 req_id = e54e53da-e71a-48f9-815c-d57359532d51
562026-09-15T04:13:54.520ZINFOdns-server (http): request completed latency_us = 678 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::c]:50652 req_id = e54e53da-e71a-48f9-815c-d57359532d51 response_code = 204 uri = /config
572026-09-15T04:13:54.695ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:57352
582026-09-15T04:13:54.697ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 920dd295-81d8-44f8-934e-8e76cf9a7c75
592026-09-15T04:13:54.697ZINFOdns-server (store): updated generation new_generation = 1 req_id = 920dd295-81d8-44f8-934e-8e76cf9a7c75
602026-09-15T04:13:54.697ZINFOdns-server (http): request completed latency_us = 211 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::b]:57352 req_id = 920dd295-81d8-44f8-934e-8e76cf9a7c75 response_code = 204 uri = /config
612026-09-15T04:13:55.388ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54518, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::a]:16807 req_id = ad1a3fcc-aa27-432e-a680-92b581433e1f
622026-09-15T04:13:55.598ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:45450
632026-09-15T04:13:55.598ZINFOdns-server (http): request completed latency_us = 250 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:45450 req_id = 99b5d144-437b-4d1a-afd7-f8b15e255be0 response_code = 200 uri = /config
642026-09-15T04:13:55.817ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22581, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::c]:24620 req_id = 196044cf-dba2-4e23-aad7-c2a71a17bb8f
652026-09-15T04:13:55.931ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:51191
662026-09-15T04:13:55.931ZINFOdns-server (http): request completed latency_us = 204 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:51191 req_id = 33ea793d-863b-4892-a807-9a94593e689f response_code = 200 uri = /config
672026-09-15T04:13:56.323ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 33782, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::b]:10186 req_id = d14d505e-5ba2-4bf1-839e-19096f63e883
682026-09-15T04:13:56.434ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:52716
692026-09-15T04:13:56.435ZINFOdns-server (http): request completed latency_us = 176 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:52716 req_id = 8374dcff-2080-4f01-ab25-261b8053461e response_code = 200 uri = /config
702026-09-15T04:14:02.098ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55584, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:50804 req_id = f58fbec2-775c-4b11-a7a4-602dea273af0
712026-09-15T04:14:12.099ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14453, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:55358 req_id = c4ec41cc-acd0-442d-bd6e-3700a6df2566
722026-09-15T04:14:22.102ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18002, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:54253 req_id = f63f82f8-b844-4e6e-940c-cc5b3aefd672
732026-09-15T04:14:32.104ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52124, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:58768 req_id = 946691ea-5475-450a-8477-2a0b6456122b
742026-09-15T04:14:42.106ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 64477, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:58855 req_id = 851d313a-a176-41da-bf67-a810dbc60578
752026-09-15T04:14:52.107ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 9060, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::d]:60866 req_id = 48d37236-f30a-4a88-9f24-498fdeb3fe27
762026-09-15T04:14:54.174ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:36262
772026-09-15T04:14:54.176ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = d7ece8dc-fe31-47de-8e8d-ee5a76d361bd
782026-09-15T04:14:54.176ZINFOdns-server (store): updated generation new_generation = 1 req_id = d7ece8dc-fe31-47de-8e8d-ee5a76d361bd
792026-09-15T04:14:54.176ZINFOdns-server (http): request completed latency_us = 252 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::c]:36262 req_id = d7ece8dc-fe31-47de-8e8d-ee5a76d361bd response_code = 204 uri = /config
802026-09-15T04:14:54.192ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:54749
812026-09-15T04:14:54.194ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 3dfafb5c-4af9-4dcf-834b-384f5f93634f
822026-09-15T04:14:54.194ZINFOdns-server (store): updated generation new_generation = 1 req_id = 3dfafb5c-4af9-4dcf-834b-384f5f93634f
832026-09-15T04:14:54.194ZINFOdns-server (http): request completed latency_us = 227 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::a]:54749 req_id = 3dfafb5c-4af9-4dcf-834b-384f5f93634f response_code = 204 uri = /config
842026-09-15T04:14:54.196ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:49386
852026-09-15T04:14:54.198ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 34981ae4-3c32-41a9-b515-31598306163b
862026-09-15T04:14:54.198ZINFOdns-server (store): updated generation new_generation = 1 req_id = 34981ae4-3c32-41a9-b515-31598306163b
872026-09-15T04:14:54.198ZINFOdns-server (http): request completed latency_us = 283 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::b]:49386 req_id = 34981ae4-3c32-41a9-b515-31598306163b response_code = 204 uri = /config
882026-09-15T04:14:55.297ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 44866, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::c]:39711 req_id = 321efa41-11cc-4a8e-8139-667ef7154cf8
892026-09-15T04:14:55.315ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54648, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::a]:12708 req_id = a33f75d6-b527-4b68-960d-95f3491f9f36
902026-09-15T04:14:55.401ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:38925
912026-09-15T04:14:55.401ZINFOdns-server (http): request completed latency_us = 181 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:38925 req_id = 8157bb9f-4901-47ad-9dda-10d844730f8c response_code = 200 uri = /config
922026-09-15T04:14:55.515ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:56541
932026-09-15T04:14:55.515ZINFOdns-server (http): request completed latency_us = 210 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:56541 req_id = faa6bca1-541b-42ba-9339-c8e360e60d1a response_code = 200 uri = /config
942026-09-15T04:14:56.054ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 13148, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::b]:3554 req_id = 339c9fbe-7075-433c-91b5-c2e7095ba3a2
952026-09-15T04:14:56.155ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:34787
962026-09-15T04:14:56.156ZINFOdns-server (http): request completed latency_us = 183 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:34787 req_id = 1669c5f0-f526-4886-acc9-bc747a985259 response_code = 200 uri = /config
972026-09-15T04:15:00.110ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 47355, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_ntp_77f0bfe8-6a21-4522-986d-cbd1700ea12f."), ), original: Query { name: Name("oxz_ntp_77f0bfe8-6a21-4522-986d-cbd1700ea12f."), query_type: A, query_class: IN, }, }, ], original: [ 44, 111, 120, 122, 95, 110, 116, 112, 95, 55, 55, 102, 48, 98, 102, 101, 56, 45, 54, 97, 50, 49, 45, 52, 53, 50, 50, 45, 57, 56, 54, 100, 45, 99, 98, 100, 49, 55, 48, 48, 101, 97, 49, 50, 102, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_ntp_77f0bfe8-6a21-4522-986d-cbd1700ea12f." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::1b]:61578 req_id = b4c25707-dae0-451c-b944-296a75da921e
982026-09-15T04:15:00.137ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 1343, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2."), ), original: Query { name: Name("oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 49, 48, 101, 50, 52, 102, 102, 48, 45, 57, 97, 99, 98, 45, 52, 101, 53, 49, 45, 97, 97, 101, 48, 45, 51, 51, 101, 99, 101, 53, 100, 49, 50, 97, 99, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::e]:50072 req_id = 48a7d21c-58d9-4d10-850c-a7d21f1ab421
992026-09-15T04:15:00.138ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 1343, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2."), ), original: Query { name: Name("oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 49, 48, 101, 50, 52, 102, 102, 48, 45, 57, 97, 99, 98, 45, 52, 101, 53, 49, 45, 97, 97, 101, 48, 45, 51, 51, 101, 99, 101, 53, 100, 49, 50, 97, 99, 50, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_10e24ff0-9acb-4e51-aae0-33ece5d12ac2." bind_address = [fd00:1122:3344:1::1]:53 peer_addr = [fd00:1122:3344:101::e]:37446 req_id = 0d5e0c31-bfae-4285-b3cc-364c2a697a21