Max level shown:
1 [ Feb 14 05:12:13 Disabled. ]
2 [ Feb 14 05:12:13 Rereading configuration. ]
3 [ Feb 14 05:12:14 Enabled. ]
4 [ Feb 14 05:12:14 Rereading configuration. ]
5 [ Feb 14 05:12:14 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/mg-ddm/pkg/ddm_method_script.sh &"). ]
6 [ Feb 14 05:12:14 Method "start" exited with status 0. ]
72026-02-14T05:12:14.906ZINFOslog-rs: signal handler waiting for context
82026-02-14T05:12:14.928ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
92026-02-14T05:12:14.928ZERROslog-rs: failed to start stats server: underlay address not found
102026-02-14T05:12:14.929ZINFOslog-rs: signal handler waiting got context
112026-02-14T05:12:14.930ZINFOslog-rs: admin: listening on [::1]:8000
122026-02-14T05:12:14.930ZINFOslog-rs: admin: server started
132026-02-14T05:12:15.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
142026-02-14T05:12:16.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
152026-02-14T05:12:17.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
162026-02-14T05:12:18.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
172026-02-14T05:12:19.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
182026-02-14T05:12:20.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
192026-02-14T05:12:21.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
202026-02-14T05:12:22.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
212026-02-14T05:12:23.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
222026-02-14T05:12:24.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
232026-02-14T05:12:25.929ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
242026-02-14T05:12:26.930ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
252026-02-14T05:12:27.930ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
262026-02-14T05:12:28.930ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
272026-02-14T05:12:29.930ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
282026-02-14T05:12:30.930ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
292026-02-14T05:12:31.930ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
302026-02-14T05:12:32.930ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
312026-02-14T05:12:33.930ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
322026-02-14T05:12:34.930ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
332026-02-14T05:12:35.930ZINFOslog-rs: [tfportrear0_0] sm initialized with addr fe80::989d:6ff:fe74:75d7 on if tfportrear0_0 index 3
342026-02-14T05:12:35.930ZDEBGslog-rs: [tfportrear0_0] starting discovery handler
35 [ Feb 14 05:12:36 Rereading configuration. ]
36 [ Feb 14 05:12:36 Executing refresh method ("/opt/oxide/mg-ddm/pkg/refresh_ctrun_child.sh -c 214"). ]
37 + CONTRACT=''
38 + getopts c: o
39 + CONTRACT=214
40 + getopts c: o
41 + [[ -n 214 ]]
42 + pgrep -c 214
43 + PARENT_PID=2291
44 + [[ -n 2291 ]]
45 + pgrep -P 2291
46 + CHILD_PID=2294
47 + [[ -n 2294 ]]
48 + kill -USR1 2294
492026-02-14T05:12:36.421ZINFOslog-rs: handling smf refresh
50 [ Feb 14 05:12:36 Method "refresh" exited with status 0. ]
512026-02-14T05:12:36.424ZINFOslog-rs: starting stats server on smf refresh
52 Feb 14 05:12:36.425 DEBG registered DTrace probes
53 Feb 14 05:12:36.425 DEBG registered endpoint, versions: all, path: /{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot
54 Feb 14 05:12:36.425 INFO listening, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328
55 Feb 14 05:12:36.425 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot
56 Feb 14 05:12:36.426 DEBG Nexus IP not provided, will use DNS to resolve it
57 Feb 14 05:12:36.426 INFO new DNS resolver, addresses: [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53], component: internal-dns-resolver, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/194594d/internal-dns/resolver/src/resolver.rs:144
58 Feb 14 05:12:36.426 DEBG starting producer registration task
59 Feb 14 05:12:36.426 INFO starting oximeter metric producer server, interval: 1s, address: [fd00:1122:3344:101::2]:8001, producer_id: b62209db-81a1-4164-ae1d-204a373cbb71, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/194594d/oximeter/producer/src/lib.rs:279
60 Feb 14 05:12:36.426 DEBG registering as oximeter producer with Nexus, registration_count: 0, component: producer-registration-task
612026-02-14T05:12:37.752ZWARNslog-rs: [tfportrear0_0] admin event in solicit state: Announce(Underlay({Ipv6Net { addr: fdd3:848b:8cbd:2c59::1, width: 64 }}))
622026-02-14T05:12:37.752ZINFOslog-rs: advertise tunnel: { TunnelOrigin { overlay_prefix: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), boundary_addr: fdd3:848b:8cbd:2c59::1, vni: 99, metric: 18446744073709551615, }, }
632026-02-14T05:12:37.752ZWARNslog-rs: [tfportrear0_0] admin event in solicit state: Announce(Tunnel({TunnelOrigin { overlay_prefix: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), boundary_addr: fdd3:848b:8cbd:2c59::1, vni: 99, metric: 18446744073709551615 }}))
642026-02-14T05:12:37.931ZINFOslog-rs: [tfportrear0_0] nbr is fe80::6013:5aff:fe23:a7e1@sock server
652026-02-14T05:12:37.931ZDEBGslog-rs: [tfportrear0_0] transition solicit -> exchange
662026-02-14T05:12:37.931ZINFOslog-rs: [tfportrear0_0] exchange: listening on [fe80::989d:6ff:fe74:75d7]:56797
672026-02-14T05:12:37.931ZINFOslog-rs: [tfportrear0_0] waiting for exchange server to start
682026-02-14T05:12:37.933ZINFOslog-rs: [tfportrear0_0] sending 5 routes to dendrite
692026-02-14T05:12:37.933ZDEBGslog-rs: [tfportrear0_0] sending to dpd host=localhost port=12224
702026-02-14T05:12:37.934ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:1:: -> fe80::6013:5aff:fe23:a7e1 on port Rear(Rear("rear0"))/0
712026-02-14T05:12:37.935ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:101:: -> fe80::6013:5aff:fe23:a7e1 on port Rear(Rear("rear0"))/0
722026-02-14T05:12:37.936ZINFOslog-rs: [tfportrear0_0] adding route fdb0:18c0:4d0d:a02a:: -> fe80::6013:5aff:fe23:a7e1 on port Rear(Rear("rear0"))/0
732026-02-14T05:12:37.936ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:2:: -> fe80::6013:5aff:fe23:a7e1 on port Rear(Rear("rear0"))/0
742026-02-14T05:12:37.936ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:3:: -> fe80::6013:5aff:fe23:a7e1 on port Rear(Rear("rear0"))/0
752026-02-14T05:12:37.936ZINFOslog-rs: [tfportrear0_0] removing routes 0 from dendrite
762026-02-14T05:12:37.936ZDEBGslog-rs: [tfportrear0_0] redistributing update to 0 peers
77 Feb 14 05:12:41.428 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("054da191-a178-41f7-88a3-dc0bff867820.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("092896e9-df71-4acb-97cc-7af5dc187ed9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8250, tv_nsec: 182731829 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
78 Feb 14 05:12:41.429 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
79 Feb 14 05:12:41.429 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
802026-02-14T05:12:53.931ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::6013:5aff:fe23:a7e1 missed solicit interval
812026-02-14T05:12:55.932ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::6013:5aff:fe23:a7e1 missed solicit interval
82 Feb 14 05:12:56.417 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
83 Feb 14 05:12:56.417 WARN failed to register with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)", delay: 20.004592346s, count: 0, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/194594d/oximeter/producer/src/lib.rs:371
84 Feb 14 05:12:56.615 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("054da191-a178-41f7-88a3-dc0bff867820.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("092896e9-df71-4acb-97cc-7af5dc187ed9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8265, tv_nsec: 383725639 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
85 Feb 14 05:12:56.616 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
86 Feb 14 05:12:56.616 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
872026-02-14T05:12:57.933ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::6013:5aff:fe23:a7e1 missed solicit interval
88 Feb 14 05:13:11.584 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
89 Feb 14 05:13:11.585 WARN failed to register with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)", delay: 35.207140663s, count: 1, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/194594d/oximeter/producer/src/lib.rs:371
90 Feb 14 05:13:12.261 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("054da191-a178-41f7-88a3-dc0bff867820.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("092896e9-df71-4acb-97cc-7af5dc187ed9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8281, tv_nsec: 64527558 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
91 Feb 14 05:13:12.262 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
92 Feb 14 05:13:12.266 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
93 Feb 14 05:13:27.267 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
94 Feb 14 05:13:27.267 WARN failed to register with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)", delay: 50.890288596s, count: 2, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/194594d/oximeter/producer/src/lib.rs:371
95 Feb 14 05:13:27.831 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("054da191-a178-41f7-88a3-dc0bff867820.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("092896e9-df71-4acb-97cc-7af5dc187ed9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8296, tv_nsec: 635442354 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
96 Feb 14 05:13:27.832 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
97 Feb 14 05:13:27.832 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
98 Feb 14 05:13:42.831 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
99 Feb 14 05:13:42.831 WARN failed to register with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)", delay: 66.456675964s, count: 3, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/194594d/oximeter/producer/src/lib.rs:371
1002026-02-14T05:13:44.170ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::6013:5aff:fe23:a7e1 missed solicit interval
101 Feb 14 05:13:47.263 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("054da191-a178-41f7-88a3-dc0bff867820.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("092896e9-df71-4acb-97cc-7af5dc187ed9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8316, tv_nsec: 15328421 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
102 Feb 14 05:13:47.281 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
103 Feb 14 05:13:47.352 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
1042026-02-14T05:13:48.770ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::6013:5aff:fe23:a7e1 missed solicit interval
1052026-02-14T05:13:54.776ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::6013:5aff:fe23:a7e1 missed solicit interval
1062026-02-14T05:13:57.056ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::6013:5aff:fe23:a7e1 missed solicit interval
107 Feb 14 05:14:02.571 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
108 Feb 14 05:14:02.591 WARN failed to register with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)", delay: 86.216221482s, count: 4, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/194594d/oximeter/producer/src/lib.rs:371
109 Feb 14 05:14:06.157 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("054da191-a178-41f7-88a3-dc0bff867820.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("092896e9-df71-4acb-97cc-7af5dc187ed9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8334, tv_nsec: 961800520 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
110 Feb 14 05:14:06.262 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
111 Feb 14 05:14:06.363 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
112 Feb 14 05:14:21.363 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
113 Feb 14 05:14:21.363 WARN failed to register with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)", delay: 104.989119435s, count: 5, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/194594d/oximeter/producer/src/lib.rs:371
114 Feb 14 05:14:26.886 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("054da191-a178-41f7-88a3-dc0bff867820.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("092896e9-df71-4acb-97cc-7af5dc187ed9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8355, tv_nsec: 692886381 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
115 Feb 14 05:14:26.898 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
116 Feb 14 05:14:26.898 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
117 Feb 14 05:14:34.798 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::a]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) }), component: producer-registration-task
118 Feb 14 05:14:34.798 WARN failed to register with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)", delay: 118.423608707s, count: 6, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/194594d/oximeter/producer/src/lib.rs:371
119 Feb 14 05:14:45.963 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("054da191-a178-41f7-88a3-dc0bff867820.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("092896e9-df71-4acb-97cc-7af5dc187ed9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("0cfdae07-f989-48d2-9467-216be9ce6ae4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8374, tv_nsec: 768878502 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
120 Feb 14 05:14:45.963 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
121 Feb 14 05:14:45.963 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
122 Feb 14 05:14:46.039 DEBG client response, result: Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "74b03264-a1f7-4721-a08b-065b3786013f", "content-length": "41", "date": "Sat, 14 Feb 2026 05:14:46 GMT"} }), component: producer-registration-task
123 Feb 14 05:14:46.039 DEBG successfully registered with Nexus, pausing until time to renew lease, wait_period: 150s, lease_duration: 600s, component: producer-registration-task
124 Feb 14 05:14:55.995 INFO accepted connection, remote_addr: [fd00:1122:3344:101::d]:44903, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:1025
125 Feb 14 05:14:56.012 INFO request completed, latency_us: 15178, response_code: 200, uri: /b62209db-81a1-4164-ae1d-204a373cbb71, method: GET, req_id: 72cb9737-a945-46b3-ac45-d9c29cc535e2, remote_addr: [fd00:1122:3344:101::d]:44903, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867
126 Feb 14 05:14:56.844 INFO request completed, latency_us: 853, response_code: 200, uri: /b62209db-81a1-4164-ae1d-204a373cbb71, method: GET, req_id: a0a56284-2385-4b08-b50c-34ddb68894ec, remote_addr: [fd00:1122:3344:101::d]:44903, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867
127 Feb 14 05:14:57.843 INFO request completed, latency_us: 899, response_code: 200, uri: /b62209db-81a1-4164-ae1d-204a373cbb71, method: GET, req_id: 0b42de2d-b7da-4da9-922f-40b8fa28abf4, remote_addr: [fd00:1122:3344:101::d]:44903, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867
128 Feb 14 05:14:58.843 INFO request completed, latency_us: 634, response_code: 200, uri: /b62209db-81a1-4164-ae1d-204a373cbb71, method: GET, req_id: 2710abbf-2c54-43cf-a367-3f17d010c7d9, remote_addr: [fd00:1122:3344:101::d]:44903, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867
129 Feb 14 05:14:59.843 INFO request completed, latency_us: 617, response_code: 200, uri: /b62209db-81a1-4164-ae1d-204a373cbb71, method: GET, req_id: 1db9b2b6-2396-4100-9ea7-392809fff8fc, remote_addr: [fd00:1122:3344:101::d]:44903, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867