Max level shown:
1 [ Jul 11 17:56:35 Enabled. ]
2 [ Jul 11 17:56:35 Rereading configuration. ]
3 [ Jul 11 17:56:36 Rereading configuration. ]
4 [ Jul 11 17:56:38 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dns-server/bin/dns-server --config-file /var/svc/manifest/site/internal_dns/config.toml --http-address [fd00:1122:3344:1::1]:5353 --dns-address [fd00:1122:3344:1::1]:53 &"). ]
5 [ Jul 11 17:56:38 Method "start" exited with status 0. ]
6 Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:1::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } }
7 note: configured to log to "/dev/stdout"
82026-07-11T17:56:38.304ZINFOdns-server: config config = Config { log: File { level: Info, path: "/dev/stdout", if_exists: Append }, dropshot: ConfigDropshot { bind_address: [fd00:1122:3344:1::1]:5353, default_request_body_max_bytes: 104857600, default_handler_task_mode: Detached, log_headers: [], compression: None }, storage: Config { storage_path: "/data/dns", keep_old_generations: 3 } } dns_config = Config { bind_address: [fd00:1122:3344:1::1]:53 }
92026-07-11T17:56:38.304ZINFOdns-server (store): opening sled database path = /data/dns
102026-07-11T17:56:38.324ZINFOdns-server (store): pruning trees for generations newer than 0
112026-07-11T17:56:38.324ZINFOdns-server (store): pruning trees for generations older than 0 keep = 3
122026-07-11T17:56:38.324ZINFOdns-server (dns): DNS server bound to address local_address = [fd00:1122:3344:1::1]:53
132026-07-11T17:56:38.325ZINFOdns-server (http): listening local_addr = [fd00:1122:3344:1::1]:5353
142026-07-11T17:56:38.473ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:1::2]:58005
152026-07-11T17:56:38.474ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = b0e72859-2da7-4e14-a140-99990585d538
162026-07-11T17:56:38.474ZINFOdns-server (store): pruning trees for generations newer than 0
172026-07-11T17:56:38.523ZINFOdns-server (store): pruning trees for generations older than 1 keep = 3
182026-07-11T17:56:38.523ZINFOdns-server (store): updated generation new_generation = 1 req_id = b0e72859-2da7-4e14-a140-99990585d538
192026-07-11T17:56:38.523ZINFOdns-server (http): request completed latency_us = 48818 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:1::2]:58005 req_id = b0e72859-2da7-4e14-a140-99990585d538 response_code = 204 uri = /config
202026-07-11T17:56:51.695ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 770, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:61833 req_id = fe6349d6-700b-4dbb-8f73-b32cc585c1eb
212026-07-11T17:56:51.701ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14116, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: AAAA, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 28, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:51243 req_id = aa2c0d8a-f240-447d-949e-08833a4902a6
222026-07-11T17:56:51.705ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 23629, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("0.pool.ntp.org."), ), original: Query { name: Name("0.pool.ntp.org."), query_type: A, query_class: IN, }, }, ], original: [ 1, 48, 4, 112, 111, 111, 108, 3, 110, 116, 112, 3, 111, 114, 103, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "0.pool.ntp.org." peer_addr = [fd00:1122:3344:101::1b]:50390 req_id = 1828fcca-bc0d-4870-be75-dda4b1e6985f
232026-07-11T17:58:23.424ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57280, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:51668 req_id = dfc6b006-b206-42f4-ae86-4ff6cb44cb8f
242026-07-11T17:58:26.773ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 63857, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9."), ), original: Query { name: Name("oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 54, 53, 97, 52, 48, 50, 97, 50, 45, 99, 51, 52, 97, 45, 52, 97, 51, 101, 45, 97, 97, 98, 51, 45, 99, 101, 99, 98, 48, 100, 56, 53, 53, 51, 101, 57, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9." peer_addr = [fd00:1122:3344:101::e]:60398 req_id = 2288e5d3-d023-4474-9651-d81418d08fa3
252026-07-11T17:58:26.784ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 63857, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9."), ), original: Query { name: Name("oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 54, 53, 97, 52, 48, 50, 97, 50, 45, 99, 51, 52, 97, 45, 52, 97, 51, 101, 45, 97, 97, 98, 51, 45, 99, 101, 99, 98, 48, 100, 56, 53, 53, 51, 101, 57, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9." peer_addr = [fd00:1122:3344:101::e]:36432 req_id = 3eeda3c3-ba16-4f04-a427-cdca2af133ce
262026-07-11T17:58:31.569ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 21657, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::b]:46848 req_id = 760c52f1-53a9-4659-9fc6-77217792fff9
272026-07-11T17:58:32.802ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 10376, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:55702 req_id = e7c9ee58-61ea-4871-be86-3af784c6d11c
282026-07-11T17:58:32.825ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45752, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:32738 req_id = 2f430c1e-e4f6-4d0a-ab21-5eedb7f2e7f8
292026-07-11T17:58:33.400ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 17186, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:64244 req_id = aa9f8603-a467-4ce1-91ba-f419008feb74
302026-07-11T17:58:43.402ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 34774, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:57236 req_id = a239f203-5a46-478e-9d5a-b859ecf53b5d
312026-07-11T17:58:53.404ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 20747, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:63781 req_id = bf6e3d64-f9d7-49ba-9cf7-85548c79cb44
322026-07-11T17:58:55.634ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 42132, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:38509 req_id = 7337af3f-a055-4f02-a215-c89003056391
332026-07-11T17:58:56.037ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:33101
342026-07-11T17:58:56.110ZINFOdns-server (http): request completed latency_us = 46719 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:33101 req_id = 0f826964-bd47-4ce3-a314-c58cc46206bb response_code = 200 uri = /config
352026-07-11T17:59:00.465ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35330, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:5365 req_id = e7da1407-11d5-4724-a790-c12ed8a1a464
362026-07-11T17:59:00.486ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:53886
372026-07-11T17:59:00.486ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = a6e05c48-dd4d-40a4-a7a2-715f940e7da4
382026-07-11T17:59:00.486ZINFOdns-server (store): updated generation new_generation = 1 req_id = a6e05c48-dd4d-40a4-a7a2-715f940e7da4
392026-07-11T17:59:00.486ZINFOdns-server (http): request completed latency_us = 460 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::a]:53886 req_id = a6e05c48-dd4d-40a4-a7a2-715f940e7da4 response_code = 204 uri = /config
402026-07-11T17:59:00.762ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:39334
412026-07-11T17:59:00.762ZINFOdns-server (http): request completed latency_us = 236 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:39334 req_id = 355f244d-f6c2-4f17-a1e0-bd1b1fd5e6bc response_code = 200 uri = /config
422026-07-11T17:59:01.661ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 15003, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:53676 req_id = d2588e12-3218-4534-8c4f-d11ac0d98f30
432026-07-11T17:59:01.905ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:38693
442026-07-11T17:59:01.906ZINFOdns-server (http): request completed latency_us = 233 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:38693 req_id = 65e3e7af-af8e-465c-b1b2-b70880205575 response_code = 200 uri = /config
452026-07-11T17:59:02.387ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35792, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::b]:51035 req_id = 3c8e511e-82c4-466b-9476-5e465ac7ce21
462026-07-11T17:59:02.521ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:43340
472026-07-11T17:59:02.522ZINFOdns-server (http): request completed latency_us = 353 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:43340 req_id = 5ac0a75c-c457-4fbd-848e-f0bf0e71afe7 response_code = 200 uri = /config
482026-07-11T17:59:03.406ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41883, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:58903 req_id = fab20872-767f-4561-b204-556ae8b0e930
492026-07-11T17:59:03.752ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 40461, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:64752 req_id = 4e863850-550e-43ee-a54e-8eb9ba931909
502026-07-11T17:59:03.882ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:35959
512026-07-11T17:59:03.882ZINFOdns-server (http): request completed latency_us = 229 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:35959 req_id = 7a8eb618-1f05-41cf-950c-eba9863c5b47 response_code = 200 uri = /config
522026-07-11T17:59:13.408ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32916, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:56927 req_id = cab99b10-6dc7-4c05-8bd2-4014e3d090a0
532026-07-11T17:59:23.409ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 27849, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:62650 req_id = 99f459a5-88d4-4175-913a-ae4c5247fd72
542026-07-11T17:59:31.485ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:38337
552026-07-11T17:59:31.485ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 005cfaaf-8dbb-44c3-84e6-e1c3d0249b7a
562026-07-11T17:59:31.485ZINFOdns-server (store): updated generation new_generation = 1 req_id = 005cfaaf-8dbb-44c3-84e6-e1c3d0249b7a
572026-07-11T17:59:31.485ZINFOdns-server (http): request completed latency_us = 233 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::b]:38337 req_id = 005cfaaf-8dbb-44c3-84e6-e1c3d0249b7a response_code = 204 uri = /config
582026-07-11T17:59:32.523ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:32796
592026-07-11T17:59:32.523ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = e04e6d54-1ea6-4f01-a068-3a043df34430
602026-07-11T17:59:32.523ZINFOdns-server (store): updated generation new_generation = 1 req_id = e04e6d54-1ea6-4f01-a068-3a043df34430
612026-07-11T17:59:32.523ZINFOdns-server (http): request completed latency_us = 292 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::a]:32796 req_id = e04e6d54-1ea6-4f01-a068-3a043df34430 response_code = 204 uri = /config
622026-07-11T17:59:32.828ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:62529
632026-07-11T17:59:32.829ZINFOdns-server (store): attempting generation update new_generation = 1 req_id = 11616225-f108-4701-afcb-628829033135
642026-07-11T17:59:32.829ZINFOdns-server (store): updated generation new_generation = 1 req_id = 11616225-f108-4701-afcb-628829033135
652026-07-11T17:59:32.829ZINFOdns-server (http): request completed latency_us = 287 local_addr = [fd00:1122:3344:1::1]:5353 method = PUT remote_addr = [fd00:1122:3344:101::c]:62529 req_id = 11616225-f108-4701-afcb-628829033135 response_code = 204 uri = /config
662026-07-11T17:59:33.193ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 39285, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::b]:37466 req_id = 7c26b01a-ac6d-4f77-91aa-731cfd5767d3
672026-07-11T17:59:33.298ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::b]:54757
682026-07-11T17:59:33.298ZINFOdns-server (http): request completed latency_us = 176 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::b]:54757 req_id = 0f5a4950-95dc-4eeb-9673-d6e9fb6369df response_code = 200 uri = /config
692026-07-11T17:59:33.411ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 13362, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:61598 req_id = 29019798-f63d-45f3-a58d-90cddc3b8d52
702026-07-11T17:59:33.778ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 65256, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::c]:40336 req_id = 38de69d8-0522-471e-b71a-e46ba0fa90a8
712026-07-11T17:59:33.839ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 9636, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-admin-keeper._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::a]:1202 req_id = b1469b21-82ed-44c7-8089-8c84a3436a75
722026-07-11T17:59:33.945ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::a]:53679
732026-07-11T17:59:33.945ZINFOdns-server (http): request completed latency_us = 263 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::a]:53679 req_id = dccacfe0-b149-49ff-860a-ae5997c1d311 response_code = 200 uri = /config
742026-07-11T17:59:33.986ZINFOdns-server (http): accepted connection local_addr = [fd00:1122:3344:1::1]:5353 remote_addr = [fd00:1122:3344:101::c]:45646
752026-07-11T17:59:33.987ZINFOdns-server (http): request completed latency_us = 180 local_addr = [fd00:1122:3344:1::1]:5353 method = GET remote_addr = [fd00:1122:3344:101::c]:45646 req_id = c2ba4f06-f945-48db-b633-6592fa002c63 response_code = 200 uri = /config
762026-07-11T17:59:43.413ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 39783, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:57833 req_id = 0f81d814-8a79-4904-95b4-ee9534da7664
772026-07-11T17:59:53.414ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35451, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-cluster-native._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, ], original: [ 26, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 99, 108, 117, 115, 116, 101, 114, 45, 110, 97, 116, 105, 118, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, flags: EdnsFlags { dnssec_ok: false, z: 0, }, max_payload: 1232, options: OPT { options: [], }, }, ), } NXDOMAIN: _clickhouse-cluster-native._tcp.control-plane.oxide.internal peer_addr = [fd00:1122:3344:101::d]:52079 req_id = 327ee712-730b-490a-98a6-ea9d81aa65da
782026-07-11T18:00:00.099ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35914, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9."), ), original: Query { name: Name("oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 54, 53, 97, 52, 48, 50, 97, 50, 45, 99, 51, 52, 97, 45, 52, 97, 51, 101, 45, 97, 97, 98, 51, 45, 99, 101, 99, 98, 48, 100, 56, 53, 53, 51, 101, 57, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9." peer_addr = [fd00:1122:3344:101::e]:54890 req_id = a1545cd7-9cb6-4e4a-8946-b5c726d3aa46
792026-07-11T18:00:00.100ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35914, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9."), ), original: Query { name: Name("oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9."), query_type: A, query_class: IN, }, }, ], original: [ 51, 111, 120, 122, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 95, 54, 53, 97, 52, 48, 50, 97, 50, 45, 99, 51, 52, 97, 45, 52, 97, 51, 101, 45, 97, 97, 98, 51, 45, 99, 101, 99, 98, 48, 100, 56, 53, 53, 51, 101, 57, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_clickhouse_65a402a2-c34a-4a3e-aab3-cecb0d8553e9." peer_addr = [fd00:1122:3344:101::e]:33316 req_id = a1bec853-e439-4d2f-95ef-7472a22580bb
802026-07-11T18:00:00.486ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14642, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_ntp_cf56f114-d8ea-4de5-9868-dbbf52c8d8e0."), ), original: Query { name: Name("oxz_ntp_cf56f114-d8ea-4de5-9868-dbbf52c8d8e0."), query_type: A, query_class: IN, }, }, ], original: [ 44, 111, 120, 122, 95, 110, 116, 112, 95, 99, 102, 53, 54, 102, 49, 49, 52, 45, 100, 56, 101, 97, 45, 52, 100, 101, 53, 45, 57, 56, 54, 56, 45, 100, 98, 98, 102, 53, 50, 99, 56, 100, 56, 101, 48, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_ntp_cf56f114-d8ea-4de5-9868-dbbf52c8d8e0." peer_addr = [fd00:1122:3344:101::1b]:52330 req_id = a35eb81c-cd9d-4783-a63a-4ce7d8db66b5
812026-07-11T18:00:00.660ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 60549, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." peer_addr = [fd00:1122:3344:101::2]:56086 req_id = d280618e-66d8-4fb8-a88c-bf1888cece68
822026-07-11T18:00:00.661ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 60549, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_switch."), ), original: Query { name: Name("oxz_switch."), query_type: A, query_class: IN, }, }, ], original: [ 10, 111, 120, 122, 95, 115, 119, 105, 116, 99, 104, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_switch." peer_addr = [fd00:1122:3344:101::2]:53693 req_id = c58ab054-858b-443d-a42b-a31050958af4
832026-07-11T18:00:00.805ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19565, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_b9c82657-e5e3-4d9d-83c3-f74e0b240f24."), ), original: Query { name: Name("oxz_cockroachdb_b9c82657-e5e3-4d9d-83c3-f74e0b240f24."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 98, 57, 99, 56, 50, 54, 53, 55, 45, 101, 53, 101, 51, 45, 52, 100, 57, 100, 45, 56, 51, 99, 51, 45, 102, 55, 52, 101, 48, 98, 50, 52, 48, 102, 50, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_b9c82657-e5e3-4d9d-83c3-f74e0b240f24." peer_addr = [fd00:1122:3344:101::6]:53036 req_id = fc18c39d-9c8c-41ae-bbad-20473b9b4fa8
842026-07-11T18:00:00.806ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 19565, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_b9c82657-e5e3-4d9d-83c3-f74e0b240f24."), ), original: Query { name: Name("oxz_cockroachdb_b9c82657-e5e3-4d9d-83c3-f74e0b240f24."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 98, 57, 99, 56, 50, 54, 53, 55, 45, 101, 53, 101, 51, 45, 52, 100, 57, 100, 45, 56, 51, 99, 51, 45, 102, 55, 52, 101, 48, 98, 50, 52, 48, 102, 50, 52, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_b9c82657-e5e3-4d9d-83c3-f74e0b240f24." peer_addr = [fd00:1122:3344:101::6]:46636 req_id = f751e0f7-5362-421a-829a-35ca0d9d8295
852026-07-11T18:00:00.832ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 65052, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_d4217a2b-6ac7-4bdd-aa78-e5146d83c949."), ), original: Query { name: Name("oxz_cockroachdb_d4217a2b-6ac7-4bdd-aa78-e5146d83c949."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 100, 52, 50, 49, 55, 97, 50, 98, 45, 54, 97, 99, 55, 45, 52, 98, 100, 100, 45, 97, 97, 55, 56, 45, 101, 53, 49, 52, 54, 100, 56, 51, 99, 57, 52, 57, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_d4217a2b-6ac7-4bdd-aa78-e5146d83c949." peer_addr = [fd00:1122:3344:101::5]:44763 req_id = 9a8427e1-70c4-49b8-a9ab-a0d8bd93d558
862026-07-11T18:00:00.832ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 65052, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_d4217a2b-6ac7-4bdd-aa78-e5146d83c949."), ), original: Query { name: Name("oxz_cockroachdb_d4217a2b-6ac7-4bdd-aa78-e5146d83c949."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 100, 52, 50, 49, 55, 97, 50, 98, 45, 54, 97, 99, 55, 45, 52, 98, 100, 100, 45, 97, 97, 55, 56, 45, 101, 53, 49, 52, 54, 100, 56, 51, 99, 57, 52, 57, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_d4217a2b-6ac7-4bdd-aa78-e5146d83c949." peer_addr = [fd00:1122:3344:101::5]:55382 req_id = a1c84710-b860-42c9-b657-4dcdc7208a2b
872026-07-11T18:00:00.890ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 8485, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_2dd419e1-abd9-4414-823e-1628f8bb3167."), ), original: Query { name: Name("oxz_cockroachdb_2dd419e1-abd9-4414-823e-1628f8bb3167."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 50, 100, 100, 52, 49, 57, 101, 49, 45, 97, 98, 100, 57, 45, 52, 52, 49, 52, 45, 56, 50, 51, 101, 45, 49, 54, 50, 56, 102, 56, 98, 98, 51, 49, 54, 55, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_2dd419e1-abd9-4414-823e-1628f8bb3167." peer_addr = [fd00:1122:3344:101::7]:63794 req_id = f1eb1fa8-b4e3-4f74-b3b0-86ba55f73efd
882026-07-11T18:00:00.891ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 8485, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_2dd419e1-abd9-4414-823e-1628f8bb3167."), ), original: Query { name: Name("oxz_cockroachdb_2dd419e1-abd9-4414-823e-1628f8bb3167."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 50, 100, 100, 52, 49, 57, 101, 49, 45, 97, 98, 100, 57, 45, 52, 52, 49, 52, 45, 56, 50, 51, 101, 45, 49, 54, 50, 56, 102, 56, 98, 98, 51, 49, 54, 55, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_2dd419e1-abd9-4414-823e-1628f8bb3167." peer_addr = [fd00:1122:3344:101::7]:35487 req_id = 4e7ce761-17ee-4d3f-ade9-57fdc0d01c35
892026-07-11T18:00:00.964ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59685, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_668265b3-a2dc-4b6f-a1ce-67afd692a1a5."), ), original: Query { name: Name("oxz_cockroachdb_668265b3-a2dc-4b6f-a1ce-67afd692a1a5."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 54, 54, 56, 50, 54, 53, 98, 51, 45, 97, 50, 100, 99, 45, 52, 98, 54, 102, 45, 97, 49, 99, 101, 45, 54, 55, 97, 102, 100, 54, 57, 50, 97, 49, 97, 53, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_668265b3-a2dc-4b6f-a1ce-67afd692a1a5." peer_addr = [fd00:1122:3344:101::3]:56558 req_id = a193c7b0-2fac-4a75-b05e-855742251579
902026-07-11T18:00:00.965ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59685, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_668265b3-a2dc-4b6f-a1ce-67afd692a1a5."), ), original: Query { name: Name("oxz_cockroachdb_668265b3-a2dc-4b6f-a1ce-67afd692a1a5."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 54, 54, 56, 50, 54, 53, 98, 51, 45, 97, 50, 100, 99, 45, 52, 98, 54, 102, 45, 97, 49, 99, 101, 45, 54, 55, 97, 102, 100, 54, 57, 50, 97, 49, 97, 53, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_668265b3-a2dc-4b6f-a1ce-67afd692a1a5." peer_addr = [fd00:1122:3344:101::3]:42848 req_id = acf3aeb2-68e3-4754-a0c3-276a5a60bc7d
912026-07-11T18:00:00.968ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22323, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_3992ae56-c56b-4bf7-8e81-deb4baec8b3d."), ), original: Query { name: Name("oxz_cockroachdb_3992ae56-c56b-4bf7-8e81-deb4baec8b3d."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 51, 57, 57, 50, 97, 101, 53, 54, 45, 99, 53, 54, 98, 45, 52, 98, 102, 55, 45, 56, 101, 56, 49, 45, 100, 101, 98, 52, 98, 97, 101, 99, 56, 98, 51, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_3992ae56-c56b-4bf7-8e81-deb4baec8b3d." peer_addr = [fd00:1122:3344:101::4]:54651 req_id = ff59ded3-8b05-40a2-9c1f-de3bec93cd2a
922026-07-11T18:00:00.969ZERROdns-server (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22323, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 0, }, queries: Queries { queries: [ LowerQuery { name: LowerName( Name("oxz_cockroachdb_3992ae56-c56b-4bf7-8e81-deb4baec8b3d."), ), original: Query { name: Name("oxz_cockroachdb_3992ae56-c56b-4bf7-8e81-deb4baec8b3d."), query_type: A, query_class: IN, }, }, ], original: [ 52, 111, 120, 122, 95, 99, 111, 99, 107, 114, 111, 97, 99, 104, 100, 98, 95, 51, 57, 57, 50, 97, 101, 53, 54, 45, 99, 53, 54, 98, 45, 52, 98, 102, 55, 45, 56, 101, 56, 49, 45, 100, 101, 98, 52, 98, 97, 101, 99, 56, 98, 51, 100, 0, 0, 1, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: None, } SERVFAIL: server is not authoritative for name: "oxz_cockroachdb_3992ae56-c56b-4bf7-8e81-deb4baec8b3d." peer_addr = [fd00:1122:3344:101::4]:33227 req_id = e79b53f1-cf2c-44f1-8eb7-7e2f330a8eda