Max level shown:
1 [ Nov 5 23:27:14 Enabled. ]
2 [ Nov 5 23:27:14 Rereading configuration. ]
3 [ Nov 5 23:27:15 Rereading configuration. ]
4 [ Nov 5 23:27:26 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Nov 5 23:27:26 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72025-11-05T23:27:26.252ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82025-11-05T23:27:26.296ZINFOtfportd: connected to dpd running softnpu-606c0be8
92025-11-05T23:27:26.297ZINFOtfportd: running in sync-only mode
102025-11-05T23:27:26.298ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112025-11-05T23:27:26.299ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132025-11-05T23:27:26.302ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( a965d965-7f72-438d-b112-74f3bdbfa34c, ), sled_id: Some( 0f6a24e9-8798-4a6f-be0a-62e430e251f9, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142025-11-05T23:27:26.302ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152025-11-05T23:27:26.305ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::580a:e9ff:fe2b:f66b->5a:0a:e9:2b:f6:6b interface = tfportrear0_0
162025-11-05T23:27:26.308ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172025-11-05T23:27:26.314ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
182025-11-05T23:27:26.319ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Nov 5 23:27:26 Rereading configuration. ]
20 [ Nov 5 23:27:26 Executing refresh method (:kill). ]
21 [ Nov 5 23:27:26 Stopping because all processes in service exited. ]
22 [ Nov 5 23:27:26 Executing stop method (:kill). ]
23 [ Nov 5 23:27:26 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Nov 5 23:27:26 Method "start" exited with status 0. ]
252025-11-05T23:27:26.877ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262025-11-05T23:27:26.918ZINFOtfportd: connected to dpd running softnpu-606c0be8
272025-11-05T23:27:26.918ZINFOtfportd: running in sync-only mode
282025-11-05T23:27:26.918ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292025-11-05T23:27:26.920ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302025-11-05T23:27:26.920ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: a965d965-7f72-438d-b112-74f3bdbfa34c, sled_id: 0f6a24e9-8798-4a6f-be0a-62e430e251f9, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, lot: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: a35b19bf-58d6-482e-ace2-58456f565824, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312025-11-05T23:27:26.920ZDEBGtfportd: created producer ID producer_id = 0554e433-a03c-41cc-a3fa-4e52e20baa91 unit = oximeter
322025-11-05T23:27:26.921ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( a965d965-7f72-438d-b112-74f3bdbfa34c, ), sled_id: Some( 0f6a24e9-8798-4a6f-be0a-62e430e251f9, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
332025-11-05T23:27:26.928ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342025-11-05T23:27:26.930ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::94a3:d4ff:fe60:a9b3->96:a3:d4:60:a9:b3 interface = tfportrear0_0
352025-11-05T23:27:26.930ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::580a:e9ff:fe2b:f66b->5a:0a:e9:2b:f6:6b interface = tfportrear0_0
362025-11-05T23:27:26.932ZDEBGtfportd: registered DTrace probes unit = producer-server
372025-11-05T23:27:26.933ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:33413 method = GET path = /{producer_id} unit = producer-server versions = all
382025-11-05T23:27:26.933ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:33413 unit = producer-server
392025-11-05T23:27:26.933ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:33413 unit = producer-server
402025-11-05T23:27:26.933ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:33413 unit = producer-server
412025-11-05T23:27:26.933ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422025-11-05T23:27:26.933ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432025-11-05T23:27:26.933ZDEBGtfportd: starting producer registration task unit = producer-server
442025-11-05T23:27:26.934ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:33413 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 0554e433-a03c-41cc-a3fa-4e52e20baa91 unit = producer-server
452025-11-05T23:27:26.934ZINFOtfportd: created oximeter producer server unit = oximeter
462025-11-05T23:27:26.934ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472025-11-05T23:27:26.934ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
482025-11-05T23:27:26.935ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492025-11-05T23:27:26.936ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2025-11-05T23:27:10.207558642Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502025-11-05T23:27:26.936ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 8285261350224492179 unit = oximeter
512025-11-05T23:27:26.936ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522025-11-05T23:27:26.938ZWARNtfportd: tfportqsfp0_0 has no addrs
532025-11-05T23:27:28.206ZINFOtfportd: added new ARP entry entry = 10.151.1.174->d6:0c:7e:54:c7:06 interface = tfportqsfp0_0
542025-11-05T23:27:28.285ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
552025-11-05T23:27:28.290ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
562025-11-05T23:27:28.294ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
572025-11-05T23:27:28.294ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
582025-11-05T23:27:28.295ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2025-11-05T23:27:10.207421769Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
592025-11-05T23:27:28.295ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 6702571997918170080 unit = oximeter
602025-11-05T23:27:28.295ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
612025-11-05T23:27:29.549ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622025-11-05T23:27:29.554ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
632025-11-05T23:27:29.557ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
642025-11-05T23:27:30.977ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652025-11-05T23:27:30.982ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
662025-11-05T23:27:30.985ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
672025-11-05T23:27:31.936ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0ae2a5d1-18e1-41cb-b6a0-df0f7691eb38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("282d3986-c55b-4482-8e40-98914e1b1218.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 868, tv_nsec: 860317183 } }) unit = producer-server
682025-11-05T23:27:31.937ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
692025-11-05T23:27:31.968ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
702025-11-05T23:27:32.412ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712025-11-05T23:27:32.416ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
722025-11-05T23:27:32.419ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
732025-11-05T23:27:33.268ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742025-11-05T23:27:33.272ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
752025-11-05T23:27:33.275ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
762025-11-05T23:27:34.553ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772025-11-05T23:27:34.559ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
782025-11-05T23:27:34.562ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
792025-11-05T23:27:35.726ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802025-11-05T23:27:35.731ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
812025-11-05T23:27:35.734ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
822025-11-05T23:27:36.897ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832025-11-05T23:27:36.902ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
842025-11-05T23:27:36.905ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
852025-11-05T23:27:36.941ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
862025-11-05T23:27:37.534ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
872025-11-05T23:27:37.539ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
882025-11-05T23:27:37.542ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
892025-11-05T23:27:38.297ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
902025-11-05T23:27:38.913ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
912025-11-05T23:27:38.919ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
922025-11-05T23:27:38.921ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
932025-11-05T23:27:39.544ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942025-11-05T23:27:39.548ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
952025-11-05T23:27:39.551ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
962025-11-05T23:27:40.243ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972025-11-05T23:27:40.249ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
982025-11-05T23:27:40.252ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
992025-11-05T23:27:41.492ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002025-11-05T23:27:41.497ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1012025-11-05T23:27:41.500ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1022025-11-05T23:27:42.822ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032025-11-05T23:27:42.827ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1042025-11-05T23:27:42.830ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1052025-11-05T23:27:43.765ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062025-11-05T23:27:43.770ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1072025-11-05T23:27:43.774ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1082025-11-05T23:27:44.898ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092025-11-05T23:27:44.903ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1102025-11-05T23:27:44.906ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1112025-11-05T23:27:46.094ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122025-11-05T23:27:46.100ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1132025-11-05T23:27:46.102ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1142025-11-05T23:27:46.788ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152025-11-05T23:27:46.793ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1162025-11-05T23:27:46.795ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1172025-11-05T23:27:46.924ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
1182025-11-05T23:27:46.950ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1192025-11-05T23:27:46.951ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 20.036323476s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
1202025-11-05T23:27:47.082ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0ae2a5d1-18e1-41cb-b6a0-df0f7691eb38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("282d3986-c55b-4482-8e40-98914e1b1218.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 884, tv_nsec: 26197249 } }) unit = producer-server
1212025-11-05T23:27:47.083ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1222025-11-05T23:27:47.113ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1232025-11-05T23:27:47.611ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1242025-11-05T23:27:47.615ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1252025-11-05T23:27:47.618ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1262025-11-05T23:27:48.276ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
1272025-11-05T23:27:48.821ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282025-11-05T23:27:48.826ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1292025-11-05T23:27:48.829ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1302025-11-05T23:27:49.794ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312025-11-05T23:27:49.799ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1322025-11-05T23:27:49.801ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1332025-11-05T23:27:50.700ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342025-11-05T23:27:50.706ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1352025-11-05T23:27:50.709ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1362025-11-05T23:27:51.651ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372025-11-05T23:27:51.655ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1382025-11-05T23:27:51.658ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1392025-11-05T23:27:52.421ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402025-11-05T23:27:52.425ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1412025-11-05T23:27:52.428ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1422025-11-05T23:27:52.968ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432025-11-05T23:27:52.974ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1442025-11-05T23:27:52.976ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1452025-11-05T23:27:53.984ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462025-11-05T23:27:53.988ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1472025-11-05T23:27:53.991ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1482025-11-05T23:27:55.315ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492025-11-05T23:27:55.321ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1502025-11-05T23:27:55.325ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1512025-11-05T23:27:55.868ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522025-11-05T23:27:55.874ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1532025-11-05T23:27:55.877ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1542025-11-05T23:27:56.899ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
1552025-11-05T23:27:57.277ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1562025-11-05T23:27:57.282ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1572025-11-05T23:27:57.285ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1582025-11-05T23:27:58.009ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1592025-11-05T23:27:58.013ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1602025-11-05T23:27:58.016ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1612025-11-05T23:27:58.252ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
1622025-11-05T23:27:59.146ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1632025-11-05T23:27:59.151ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1642025-11-05T23:27:59.154ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1652025-11-05T23:28:00.151ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1662025-11-05T23:28:00.157ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1672025-11-05T23:28:00.160ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1682025-11-05T23:28:01.080ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1692025-11-05T23:28:01.085ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1702025-11-05T23:28:01.089ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1712025-11-05T23:28:02.221ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1722025-11-05T23:28:02.521ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1732025-11-05T23:28:02.561ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1742025-11-05T23:28:02.622ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 35.748905199s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
1752025-11-05T23:28:02.695ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1762025-11-05T23:28:03.318ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0ae2a5d1-18e1-41cb-b6a0-df0f7691eb38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("282d3986-c55b-4482-8e40-98914e1b1218.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 900, tv_nsec: 305440293 } }) unit = producer-server
1772025-11-05T23:28:03.319ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1782025-11-05T23:28:03.381ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1792025-11-05T23:28:03.677ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1802025-11-05T23:28:03.684ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1812025-11-05T23:28:03.695ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1822025-11-05T23:28:05.296ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1832025-11-05T23:28:05.300ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1842025-11-05T23:28:05.309ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1852025-11-05T23:28:06.158ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862025-11-05T23:28:06.165ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1872025-11-05T23:28:06.169ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1882025-11-05T23:28:06.879ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
1892025-11-05T23:28:07.225ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1902025-11-05T23:28:07.231ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1912025-11-05T23:28:07.235ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1922025-11-05T23:28:08.228ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
1932025-11-05T23:28:08.610ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1942025-11-05T23:28:08.614ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1952025-11-05T23:28:08.616ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1962025-11-05T23:28:09.560ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1972025-11-05T23:28:09.565ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
1982025-11-05T23:28:09.568ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1992025-11-05T23:28:10.991ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002025-11-05T23:28:10.996ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2012025-11-05T23:28:11.000ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2022025-11-05T23:28:12.043ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032025-11-05T23:28:12.048ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2042025-11-05T23:28:12.050ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2052025-11-05T23:28:12.620ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2062025-11-05T23:28:12.627ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2072025-11-05T23:28:12.630ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2082025-11-05T23:28:16.353ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092025-11-05T23:28:16.361ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2102025-11-05T23:28:16.377ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2112025-11-05T23:28:16.861ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
2122025-11-05T23:28:17.346ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2132025-11-05T23:28:17.353ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2142025-11-05T23:28:17.359ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2152025-11-05T23:28:17.994ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2162025-11-05T23:28:18.001ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2172025-11-05T23:28:18.005ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2182025-11-05T23:28:18.235ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
2192025-11-05T23:28:18.347ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2202025-11-05T23:28:18.385ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 51.552236022s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
2212025-11-05T23:28:18.846ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2222025-11-05T23:28:18.853ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2232025-11-05T23:28:18.857ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2242025-11-05T23:28:19.708ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0ae2a5d1-18e1-41cb-b6a0-df0f7691eb38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("282d3986-c55b-4482-8e40-98914e1b1218.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 916, tv_nsec: 731700764 } }) unit = producer-server
2252025-11-05T23:28:19.709ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2262025-11-05T23:28:19.775ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2272025-11-05T23:28:19.806ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2282025-11-05T23:28:19.812ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2292025-11-05T23:28:19.816ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2302025-11-05T23:28:20.939ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2312025-11-05T23:28:20.946ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2322025-11-05T23:28:20.950ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2332025-11-05T23:28:21.595ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342025-11-05T23:28:21.602ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2352025-11-05T23:28:21.606ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2362025-11-05T23:28:22.783ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372025-11-05T23:28:22.788ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2382025-11-05T23:28:22.791ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2392025-11-05T23:28:23.939ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402025-11-05T23:28:23.944ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2412025-11-05T23:28:23.947ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2422025-11-05T23:28:24.626ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2432025-11-05T23:28:24.631ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2442025-11-05T23:28:24.634ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2452025-11-05T23:28:25.496ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462025-11-05T23:28:25.501ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2472025-11-05T23:28:25.503ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2482025-11-05T23:28:26.374ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2492025-11-05T23:28:26.380ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2502025-11-05T23:28:26.384ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2512025-11-05T23:28:26.864ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
2522025-11-05T23:28:27.463ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2532025-11-05T23:28:27.468ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2542025-11-05T23:28:27.471ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2552025-11-05T23:28:28.230ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2562025-11-05T23:28:28.235ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2572025-11-05T23:28:28.238ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
2582025-11-05T23:28:28.239ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2592025-11-05T23:28:28.939ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2602025-11-05T23:28:28.944ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2612025-11-05T23:28:28.947ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2622025-11-05T23:28:30.321ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2632025-11-05T23:28:30.326ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2642025-11-05T23:28:30.329ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2652025-11-05T23:28:31.267ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2662025-11-05T23:28:31.272ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2672025-11-05T23:28:31.275ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2682025-11-05T23:28:31.886ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692025-11-05T23:28:31.892ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2702025-11-05T23:28:31.895ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2712025-11-05T23:28:33.136ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722025-11-05T23:28:33.141ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2732025-11-05T23:28:33.145ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2742025-11-05T23:28:33.986ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2752025-11-05T23:28:33.992ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2762025-11-05T23:28:33.995ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2772025-11-05T23:28:34.776ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2782025-11-05T23:28:34.778ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 67.944071931s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
2792025-11-05T23:28:35.183ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2802025-11-05T23:28:35.187ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2812025-11-05T23:28:35.190ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2822025-11-05T23:28:36.006ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0ae2a5d1-18e1-41cb-b6a0-df0f7691eb38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("282d3986-c55b-4482-8e40-98914e1b1218.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 933, tv_nsec: 30393899 } }) unit = producer-server
2832025-11-05T23:28:36.007ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2842025-11-05T23:28:36.039ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2852025-11-05T23:28:36.498ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2862025-11-05T23:28:36.503ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2872025-11-05T23:28:36.506ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2882025-11-05T23:28:36.866ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
2892025-11-05T23:28:37.042ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2902025-11-05T23:28:37.047ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2912025-11-05T23:28:37.049ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2922025-11-05T23:28:37.635ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2932025-11-05T23:28:37.640ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2942025-11-05T23:28:37.644ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2952025-11-05T23:28:38.241ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
2962025-11-05T23:28:38.754ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2972025-11-05T23:28:38.759ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
2982025-11-05T23:28:38.762ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2992025-11-05T23:28:39.702ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3002025-11-05T23:28:39.707ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3012025-11-05T23:28:39.710ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3022025-11-05T23:28:40.910ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3032025-11-05T23:28:40.915ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3042025-11-05T23:28:40.918ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3052025-11-05T23:28:41.843ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3062025-11-05T23:28:41.848ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3072025-11-05T23:28:41.851ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3082025-11-05T23:28:42.524ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3092025-11-05T23:28:42.530ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3102025-11-05T23:28:42.533ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3112025-11-05T23:28:43.660ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3122025-11-05T23:28:43.665ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3132025-11-05T23:28:43.668ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3142025-11-05T23:28:44.836ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3152025-11-05T23:28:44.842ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3162025-11-05T23:28:44.845ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3172025-11-05T23:28:45.492ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3182025-11-05T23:28:45.501ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3192025-11-05T23:28:45.505ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3202025-11-05T23:28:46.870ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
3212025-11-05T23:28:46.963ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3222025-11-05T23:28:46.972ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3232025-11-05T23:28:46.981ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3242025-11-05T23:28:48.136ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3252025-11-05T23:28:48.144ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3262025-11-05T23:28:48.150ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3272025-11-05T23:28:48.247ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
3282025-11-05T23:28:49.068ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3292025-11-05T23:28:49.079ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3302025-11-05T23:28:49.085ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3312025-11-05T23:28:49.645ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3322025-11-05T23:28:49.653ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3332025-11-05T23:28:49.658ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3342025-11-05T23:28:51.040ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3352025-11-05T23:28:51.042ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 84.207855604s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
3362025-11-05T23:28:51.087ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3372025-11-05T23:28:51.094ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3382025-11-05T23:28:51.098ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3392025-11-05T23:28:52.314ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3402025-11-05T23:28:52.336ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3412025-11-05T23:28:52.341ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3422025-11-05T23:28:53.531ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3432025-11-05T23:28:53.539ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3442025-11-05T23:28:53.543ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3452025-11-05T23:28:54.188ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3462025-11-05T23:28:54.194ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3472025-11-05T23:28:54.199ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3482025-11-05T23:28:54.917ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3492025-11-05T23:28:54.925ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3502025-11-05T23:28:54.930ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3512025-11-05T23:28:55.562ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0ae2a5d1-18e1-41cb-b6a0-df0f7691eb38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("282d3986-c55b-4482-8e40-98914e1b1218.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 952, tv_nsec: 585789428 } }) unit = producer-server
3522025-11-05T23:28:55.563ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
3532025-11-05T23:28:55.703ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3542025-11-05T23:28:56.033ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3552025-11-05T23:28:56.038ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3562025-11-05T23:28:56.042ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3572025-11-05T23:28:56.599ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3582025-11-05T23:28:56.605ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3592025-11-05T23:28:56.607ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3602025-11-05T23:28:56.874ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
3612025-11-05T23:28:57.645ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3622025-11-05T23:28:57.650ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3632025-11-05T23:28:57.653ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3642025-11-05T23:28:58.249ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
3652025-11-05T23:28:59.153ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3662025-11-05T23:28:59.158ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3672025-11-05T23:28:59.162ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3682025-11-05T23:29:00.488ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3692025-11-05T23:29:00.493ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3702025-11-05T23:29:00.496ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3712025-11-05T23:29:01.428ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3722025-11-05T23:29:01.433ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3732025-11-05T23:29:01.436ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3742025-11-05T23:29:02.938ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3752025-11-05T23:29:02.944ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3762025-11-05T23:29:02.947ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3772025-11-05T23:29:03.808ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3782025-11-05T23:29:03.815ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3792025-11-05T23:29:03.824ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3802025-11-05T23:29:04.514ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3812025-11-05T23:29:04.522ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3822025-11-05T23:29:04.528ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3832025-11-05T23:29:05.485ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3842025-11-05T23:29:05.491ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3852025-11-05T23:29:05.495ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3862025-11-05T23:29:06.878ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
3872025-11-05T23:29:06.924ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3882025-11-05T23:29:06.931ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3892025-11-05T23:29:06.934ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3902025-11-05T23:29:08.253ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
3912025-11-05T23:29:08.256ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3922025-11-05T23:29:08.261ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3932025-11-05T23:29:08.263ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3942025-11-05T23:29:09.139ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3952025-11-05T23:29:09.145ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3962025-11-05T23:29:09.148ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3972025-11-05T23:29:10.049ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3982025-11-05T23:29:10.055ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
3992025-11-05T23:29:10.059ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4002025-11-05T23:29:10.612ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4012025-11-05T23:29:10.617ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4022025-11-05T23:29:10.620ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4032025-11-05T23:29:10.706ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4042025-11-05T23:29:10.707ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 103.870808892s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
4052025-11-05T23:29:11.925ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4062025-11-05T23:29:11.931ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4072025-11-05T23:29:11.935ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4082025-11-05T23:29:13.213ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4092025-11-05T23:29:13.219ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4102025-11-05T23:29:13.223ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4112025-11-05T23:29:13.984ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4122025-11-05T23:29:13.990ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4132025-11-05T23:29:13.994ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4142025-11-05T23:29:14.734ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4152025-11-05T23:29:14.739ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4162025-11-05T23:29:14.742ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4172025-11-05T23:29:15.268ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4182025-11-05T23:29:15.274ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4192025-11-05T23:29:15.277ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4202025-11-05T23:29:16.624ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4212025-11-05T23:29:16.628ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4222025-11-05T23:29:16.632ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4232025-11-05T23:29:16.882ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
4242025-11-05T23:29:17.946ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4252025-11-05T23:29:17.952ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4262025-11-05T23:29:17.956ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4272025-11-05T23:29:18.255ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
4282025-11-05T23:29:18.991ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4292025-11-05T23:29:18.997ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4302025-11-05T23:29:19.001ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4312025-11-05T23:29:20.150ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4322025-11-05T23:29:20.156ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4332025-11-05T23:29:20.160ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4342025-11-05T23:29:21.329ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4352025-11-05T23:29:21.335ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4362025-11-05T23:29:21.338ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4372025-11-05T23:29:21.930ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0ae2a5d1-18e1-41cb-b6a0-df0f7691eb38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("282d3986-c55b-4482-8e40-98914e1b1218.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 978, tv_nsec: 953051236 } }) unit = producer-server
4382025-11-05T23:29:21.931ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
4392025-11-05T23:29:21.961ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
4402025-11-05T23:29:21.966ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::c]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) }) unit = producer-server
4412025-11-05T23:29:21.967ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 115.13143568s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
4422025-11-05T23:29:22.833ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4432025-11-05T23:29:22.841ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4442025-11-05T23:29:22.844ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4452025-11-05T23:29:23.448ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4462025-11-05T23:29:23.454ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4472025-11-05T23:29:23.458ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4482025-11-05T23:29:24.728ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4492025-11-05T23:29:24.735ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4502025-11-05T23:29:24.739ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4512025-11-05T23:29:25.663ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4522025-11-05T23:29:25.669ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4532025-11-05T23:29:25.672ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4542025-11-05T23:29:26.612ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4552025-11-05T23:29:26.619ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4562025-11-05T23:29:26.623ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4572025-11-05T23:29:26.884ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
4582025-11-05T23:29:27.351ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4592025-11-05T23:29:27.357ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4602025-11-05T23:29:27.360ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4612025-11-05T23:29:28.077ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4622025-11-05T23:29:28.083ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4632025-11-05T23:29:28.087ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4642025-11-05T23:29:28.258ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
4652025-11-05T23:29:28.832ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4662025-11-05T23:29:28.840ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4672025-11-05T23:29:28.845ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4682025-11-05T23:29:29.602ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4692025-11-05T23:29:29.610ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4702025-11-05T23:29:29.614ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4712025-11-05T23:29:30.342ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4722025-11-05T23:29:30.349ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4732025-11-05T23:29:30.352ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4742025-11-05T23:29:31.183ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4752025-11-05T23:29:31.189ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4762025-11-05T23:29:31.193ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4772025-11-05T23:29:32.469ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4782025-11-05T23:29:32.475ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4792025-11-05T23:29:32.479ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4802025-11-05T23:29:33.757ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4812025-11-05T23:29:33.763ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4822025-11-05T23:29:33.767ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4832025-11-05T23:29:34.883ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4842025-11-05T23:29:34.889ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4852025-11-05T23:29:34.893ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4862025-11-05T23:29:35.796ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4872025-11-05T23:29:35.802ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4882025-11-05T23:29:35.805ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4892025-11-05T23:29:36.887ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
4902025-11-05T23:29:37.052ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4912025-11-05T23:29:37.058ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4922025-11-05T23:29:37.061ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4932025-11-05T23:29:38.041ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4942025-11-05T23:29:38.047ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4952025-11-05T23:29:38.053ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4962025-11-05T23:29:38.262ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
4972025-11-05T23:29:39.335ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4982025-11-05T23:29:39.344ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
4992025-11-05T23:29:39.348ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5002025-11-05T23:29:39.943ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5012025-11-05T23:29:39.949ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5022025-11-05T23:29:39.952ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5032025-11-05T23:29:40.712ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5042025-11-05T23:29:40.717ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5052025-11-05T23:29:40.720ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5062025-11-05T23:29:41.214ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0ae2a5d1-18e1-41cb-b6a0-df0f7691eb38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("282d3986-c55b-4482-8e40-98914e1b1218.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dc8b9638-6b81-4cc0-8cca-57f57b326434.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 998, tv_nsec: 237058350 } }) unit = producer-server
5072025-11-05T23:29:41.215ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
5082025-11-05T23:29:41.246ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
5092025-11-05T23:29:41.261ZDEBGtfportd (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "3a67f2f9-cdf7-4820-9049-5f2f325f1587", "content-length": "41", "date": "Wed, 05 Nov 2025 23:29:41 GMT"} }) unit = producer-server
5102025-11-05T23:29:41.262ZDEBGtfportd (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s unit = producer-server wait_period = 150s
5112025-11-05T23:29:42.077ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5122025-11-05T23:29:42.083ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5132025-11-05T23:29:42.087ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5142025-11-05T23:29:42.613ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5152025-11-05T23:29:42.618ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5162025-11-05T23:29:42.621ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5172025-11-05T23:29:43.767ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5182025-11-05T23:29:43.772ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5192025-11-05T23:29:43.775ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5202025-11-05T23:29:44.826ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5212025-11-05T23:29:44.831ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5222025-11-05T23:29:44.834ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5232025-11-05T23:29:45.760ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5242025-11-05T23:29:45.765ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5252025-11-05T23:29:45.769ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5262025-11-05T23:29:46.731ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5272025-11-05T23:29:46.737ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5282025-11-05T23:29:46.740ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5292025-11-05T23:29:46.890ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
5302025-11-05T23:29:47.680ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5312025-11-05T23:29:47.685ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5322025-11-05T23:29:47.688ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5332025-11-05T23:29:48.265ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
5342025-11-05T23:29:48.456ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5352025-11-05T23:29:48.461ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5362025-11-05T23:29:48.465ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5372025-11-05T23:29:49.062ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5382025-11-05T23:29:49.068ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5392025-11-05T23:29:49.071ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5402025-11-05T23:29:49.708ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5412025-11-05T23:29:49.714ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5422025-11-05T23:29:49.718ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5432025-11-05T23:29:51.185ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5442025-11-05T23:29:51.191ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5452025-11-05T23:29:51.195ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5462025-11-05T23:29:52.121ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5472025-11-05T23:29:52.126ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5482025-11-05T23:29:52.130ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5492025-11-05T23:29:53.532ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5502025-11-05T23:29:53.538ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5512025-11-05T23:29:53.541ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5522025-11-05T23:29:54.847ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5532025-11-05T23:29:54.852ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5542025-11-05T23:29:54.856ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5552025-11-05T23:29:55.919ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5562025-11-05T23:29:55.925ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5572025-11-05T23:29:55.928ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5582025-11-05T23:29:56.743ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5592025-11-05T23:29:56.754ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5602025-11-05T23:29:56.757ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5612025-11-05T23:29:56.893ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8285261350224492179 n_samples = 6 unit = oximeter
5622025-11-05T23:29:58.233ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5632025-11-05T23:29:58.246ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5642025-11-05T23:29:58.249ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5652025-11-05T23:29:58.267ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6702571997918170080 n_samples = 6 unit = oximeter
5662025-11-05T23:29:59.030ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5672025-11-05T23:29:59.039ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5682025-11-05T23:29:59.044ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5692025-11-05T23:30:00.423ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5702025-11-05T23:30:00.430ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::580a:e9ff:fe2b:f66b]
5712025-11-05T23:30:00.435ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]