Max level shown:
1 [ Nov 18 15:12:59 Enabled. ]
2 [ Nov 18 15:12:59 Rereading configuration. ]
3 [ Nov 18 15:13:00 Rereading configuration. ]
4 [ Nov 18 15:13:11 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Nov 18 15:13:11 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72025-11-18T15:13:11.639ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82025-11-18T15:13:11.679ZINFOtfportd: connected to dpd running softnpu-606c0be8
92025-11-18T15:13:11.680ZINFOtfportd: running in sync-only mode
102025-11-18T15:13:11.682ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112025-11-18T15:13:11.682ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132025-11-18T15:13:11.684ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 1356bd86-d0e6-44d9-9c7a-51eac89139f2, ), sled_id: Some( 01ec4c5a-1657-4574-9265-dda881ba6d65, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142025-11-18T15:13:11.684ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152025-11-18T15:13:11.688ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::cc33:bfff:fe58:fe04->ce:33:bf:58:fe:04 interface = tfportrear0_0
162025-11-18T15:13:11.694ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172025-11-18T15:13:11.698ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
182025-11-18T15:13:11.703ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Nov 18 15:13:12 Rereading configuration. ]
20 [ Nov 18 15:13:12 Executing refresh method (:kill). ]
21 [ Nov 18 15:13:12 Stopping because all processes in service exited. ]
22 [ Nov 18 15:13:12 Executing stop method (:kill). ]
23 [ Nov 18 15:13:12 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Nov 18 15:13:12 Method "start" exited with status 0. ]
252025-11-18T15:13:12.237ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262025-11-18T15:13:12.276ZINFOtfportd: connected to dpd running softnpu-606c0be8
272025-11-18T15:13:12.277ZINFOtfportd: running in sync-only mode
282025-11-18T15:13:12.277ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292025-11-18T15:13:12.278ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302025-11-18T15:13:12.279ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 1356bd86-d0e6-44d9-9c7a-51eac89139f2, sled_id: 01ec4c5a-1657-4574-9265-dda881ba6d65, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, lot: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 40cceb87-cfca-485d-bbc0-9dac89c218e6, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312025-11-18T15:13:12.279ZDEBGtfportd: created producer ID producer_id = 52761a5b-097b-49f0-b5df-11eec0e94c71 unit = oximeter
322025-11-18T15:13:12.280ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 1356bd86-d0e6-44d9-9c7a-51eac89139f2, ), sled_id: Some( 01ec4c5a-1657-4574-9265-dda881ba6d65, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
332025-11-18T15:13:12.287ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342025-11-18T15:13:12.288ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::cc33:bfff:fe58:fe04->ce:33:bf:58:fe:04 interface = tfportrear0_0
352025-11-18T15:13:12.289ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::a873:5fff:feb2:2e39->aa:73:5f:b2:2e:39 interface = tfportrear0_0
362025-11-18T15:13:12.290ZDEBGtfportd: registered DTrace probes unit = producer-server
372025-11-18T15:13:12.290ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:35719 method = GET path = /{producer_id} unit = producer-server versions = all
382025-11-18T15:13:12.290ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:35719 unit = producer-server
392025-11-18T15:13:12.290ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:35719 unit = producer-server
402025-11-18T15:13:12.290ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:35719 unit = producer-server
412025-11-18T15:13:12.290ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422025-11-18T15:13:12.290ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432025-11-18T15:13:12.291ZDEBGtfportd: starting producer registration task unit = producer-server
442025-11-18T15:13:12.291ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:35719 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 52761a5b-097b-49f0-b5df-11eec0e94c71 unit = producer-server
452025-11-18T15:13:12.291ZINFOtfportd: created oximeter producer server unit = oximeter
462025-11-18T15:13:12.291ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472025-11-18T15:13:12.293ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
482025-11-18T15:13:12.293ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492025-11-18T15:13:12.294ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2025-11-18T15:12:55.606588952Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502025-11-18T15:13:12.294ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 119716041416039592 unit = oximeter
512025-11-18T15:13:12.294ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522025-11-18T15:13:12.296ZWARNtfportd: tfportqsfp0_0 has no addrs
532025-11-18T15:13:13.065ZINFOtfportd: added new ARP entry entry = 10.151.1.174->7e:03:60:01:91:d3 interface = tfportqsfp0_0
542025-11-18T15:13:13.532ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
552025-11-18T15:13:13.537ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
562025-11-18T15:13:13.539ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
572025-11-18T15:13:13.539ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
582025-11-18T15:13:13.541ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2025-11-18T15:12:55.606453848Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
592025-11-18T15:13:13.541ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 15694099324149499291 unit = oximeter
602025-11-18T15:13:13.541ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
612025-11-18T15:13:14.207ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622025-11-18T15:13:14.212ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
632025-11-18T15:13:14.215ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
642025-11-18T15:13:14.802ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652025-11-18T15:13:14.806ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
662025-11-18T15:13:14.808ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
672025-11-18T15:13:16.258ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682025-11-18T15:13:16.263ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
692025-11-18T15:13:16.266ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
702025-11-18T15:13:17.056ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712025-11-18T15:13:17.061ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
722025-11-18T15:13:17.064ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
732025-11-18T15:13:17.294ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3bd16592-14ca-4d00-959b-9efb0f600b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6b993aa7-f2e0-4e9a-b1d1-ea7a8ea320d6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 3793, tv_nsec: 599857832 } }) unit = producer-server
742025-11-18T15:13:17.295ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
752025-11-18T15:13:17.326ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
762025-11-18T15:13:17.865ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772025-11-18T15:13:17.869ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
782025-11-18T15:13:17.872ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
792025-11-18T15:13:18.777ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802025-11-18T15:13:18.782ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
812025-11-18T15:13:18.785ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
822025-11-18T15:13:19.750ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832025-11-18T15:13:19.754ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
842025-11-18T15:13:19.757ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
852025-11-18T15:13:20.981ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862025-11-18T15:13:20.986ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
872025-11-18T15:13:20.989ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
882025-11-18T15:13:22.179ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
892025-11-18T15:13:22.184ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
902025-11-18T15:13:22.186ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
912025-11-18T15:13:22.298ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter
922025-11-18T15:13:23.543ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15694099324149499291 n_samples = 6 unit = oximeter
932025-11-18T15:13:23.629ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942025-11-18T15:13:23.635ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
952025-11-18T15:13:23.638ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
962025-11-18T15:13:24.636ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972025-11-18T15:13:24.640ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
982025-11-18T15:13:24.644ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
992025-11-18T15:13:25.867ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002025-11-18T15:13:25.872ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1012025-11-18T15:13:25.875ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1022025-11-18T15:13:26.989ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032025-11-18T15:13:26.994ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1042025-11-18T15:13:26.996ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1052025-11-18T15:13:27.971ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062025-11-18T15:13:27.975ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1072025-11-18T15:13:27.978ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1082025-11-18T15:13:28.564ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092025-11-18T15:13:28.568ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1102025-11-18T15:13:28.571ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1112025-11-18T15:13:29.203ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122025-11-18T15:13:29.208ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1132025-11-18T15:13:29.211ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1142025-11-18T15:13:30.713ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152025-11-18T15:13:30.717ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1162025-11-18T15:13:30.721ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1172025-11-18T15:13:31.446ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1182025-11-18T15:13:31.450ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1192025-11-18T15:13:31.453ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1202025-11-18T15:13:32.468ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter
1212025-11-18T15:13:32.495ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1222025-11-18T15:13:32.497ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 20.038142021s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
1232025-11-18T15:13:32.631ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3bd16592-14ca-4d00-959b-9efb0f600b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6b993aa7-f2e0-4e9a-b1d1-ea7a8ea320d6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 3808, tv_nsec: 769686319 } }) unit = producer-server
1242025-11-18T15:13:32.632ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1252025-11-18T15:13:32.684ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1262025-11-18T15:13:32.875ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1272025-11-18T15:13:32.881ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1282025-11-18T15:13:32.885ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1292025-11-18T15:13:33.715ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15694099324149499291 n_samples = 6 unit = oximeter
1302025-11-18T15:13:34.019ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312025-11-18T15:13:34.025ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1322025-11-18T15:13:34.028ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1332025-11-18T15:13:35.277ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342025-11-18T15:13:35.284ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1352025-11-18T15:13:35.300ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1362025-11-18T15:13:36.497ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372025-11-18T15:13:36.514ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1382025-11-18T15:13:36.542ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1392025-11-18T15:13:37.520ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402025-11-18T15:13:37.526ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1412025-11-18T15:13:37.532ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1422025-11-18T15:13:38.143ZINFOtfportd: removed stale NDP entry :fe80::a873:5fff:feb2:2e39->aa:73:5f:b2:2e:39
1432025-11-18T15:13:38.446ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1442025-11-18T15:13:38.452ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1452025-11-18T15:13:38.455ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1462025-11-18T15:13:38.999ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::a873:5fff:feb2:2e39->aa:73:5f:b2:2e:39 interface = tfportrear0_0
1472025-11-18T15:13:39.037ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1482025-11-18T15:13:39.042ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1492025-11-18T15:13:39.044ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1502025-11-18T15:13:40.449ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1512025-11-18T15:13:40.454ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1522025-11-18T15:13:40.458ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1532025-11-18T15:13:41.639ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1542025-11-18T15:13:41.644ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1552025-11-18T15:13:41.647ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1562025-11-18T15:13:42.496ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter
1572025-11-18T15:13:42.891ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1582025-11-18T15:13:42.896ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1592025-11-18T15:13:42.899ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1602025-11-18T15:13:43.466ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1612025-11-18T15:13:43.473ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1622025-11-18T15:13:43.476ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1632025-11-18T15:13:43.745ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15694099324149499291 n_samples = 6 unit = oximeter
1642025-11-18T15:13:44.806ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1652025-11-18T15:13:44.812ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1662025-11-18T15:13:44.815ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1672025-11-18T15:13:45.609ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1682025-11-18T15:13:45.620ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1692025-11-18T15:13:45.624ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1702025-11-18T15:13:46.598ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1712025-11-18T15:13:46.607ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1722025-11-18T15:13:46.611ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1732025-11-18T15:13:48.543ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1742025-11-18T15:13:48.558ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1752025-11-18T15:13:48.560ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1762025-11-18T15:13:51.437ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 38.914127588s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
1772025-11-18T15:13:51.507ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1782025-11-18T15:13:52.187ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3bd16592-14ca-4d00-959b-9efb0f600b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6b993aa7-f2e0-4e9a-b1d1-ea7a8ea320d6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 3828, tv_nsec: 271875866 } }) unit = producer-server
1792025-11-18T15:13:52.190ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1802025-11-18T15:13:52.405ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1812025-11-18T15:13:52.441ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1822025-11-18T15:13:52.454ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1832025-11-18T15:13:52.461ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1842025-11-18T15:13:52.549ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter
1852025-11-18T15:13:53.776ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15694099324149499291 n_samples = 6 unit = oximeter
1862025-11-18T15:13:53.784ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1872025-11-18T15:13:53.792ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1882025-11-18T15:13:53.798ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1892025-11-18T15:13:54.899ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1902025-11-18T15:13:54.906ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1912025-11-18T15:13:54.914ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1922025-11-18T15:13:56.332ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1932025-11-18T15:13:56.339ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1942025-11-18T15:13:56.344ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1952025-11-18T15:13:57.770ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1962025-11-18T15:13:57.777ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
1972025-11-18T15:13:57.782ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1982025-11-18T15:13:59.166ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1992025-11-18T15:13:59.172ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2002025-11-18T15:13:59.176ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2012025-11-18T15:14:00.332ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2022025-11-18T15:14:00.338ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2032025-11-18T15:14:00.342ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2042025-11-18T15:14:01.482ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2052025-11-18T15:14:01.492ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2062025-11-18T15:14:01.499ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2072025-11-18T15:14:02.439ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2082025-11-18T15:14:02.451ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2092025-11-18T15:14:02.459ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2102025-11-18T15:14:02.642ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter
2112025-11-18T15:14:03.819ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15694099324149499291 n_samples = 6 unit = oximeter
2122025-11-18T15:14:03.938ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2132025-11-18T15:14:03.947ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2142025-11-18T15:14:03.953ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2152025-11-18T15:14:05.072ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2162025-11-18T15:14:05.085ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2172025-11-18T15:14:05.091ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2182025-11-18T15:14:06.371ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2192025-11-18T15:14:06.379ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2202025-11-18T15:14:06.384ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2212025-11-18T15:14:07.349ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2222025-11-18T15:14:07.489ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 54.963314572s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
2232025-11-18T15:14:07.706ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2242025-11-18T15:14:07.713ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2252025-11-18T15:14:08.876ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2262025-11-18T15:14:08.930ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3bd16592-14ca-4d00-959b-9efb0f600b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6b993aa7-f2e0-4e9a-b1d1-ea7a8ea320d6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 3844, tv_nsec: 945540092 } }) unit = producer-server
2272025-11-18T15:14:09.120ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2282025-11-18T15:14:09.310ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2292025-11-18T15:14:09.991ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2302025-11-18T15:14:10.001ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2312025-11-18T15:14:10.054ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2322025-11-18T15:14:11.198ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2332025-11-18T15:14:11.205ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2342025-11-18T15:14:11.209ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2352025-11-18T15:14:12.647ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter
2362025-11-18T15:14:12.678ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372025-11-18T15:14:12.682ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2382025-11-18T15:14:12.685ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2392025-11-18T15:14:13.825ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15694099324149499291 n_samples = 6 unit = oximeter
2402025-11-18T15:14:13.854ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2412025-11-18T15:14:13.860ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2422025-11-18T15:14:13.898ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2432025-11-18T15:14:15.392ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2442025-11-18T15:14:15.397ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2452025-11-18T15:14:15.400ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2462025-11-18T15:14:16.742ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2472025-11-18T15:14:16.745ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2482025-11-18T15:14:16.748ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2492025-11-18T15:14:17.985ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2502025-11-18T15:14:17.990ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2512025-11-18T15:14:17.993ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2522025-11-18T15:14:19.024ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2532025-11-18T15:14:19.029ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2542025-11-18T15:14:19.032ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2552025-11-18T15:14:19.945ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2562025-11-18T15:14:19.949ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2572025-11-18T15:14:19.952ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2582025-11-18T15:14:20.762ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2592025-11-18T15:14:20.766ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2602025-11-18T15:14:20.769ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2612025-11-18T15:14:21.883ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2622025-11-18T15:14:21.889ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2632025-11-18T15:14:21.893ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2642025-11-18T15:14:22.651ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter
2652025-11-18T15:14:23.112ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2662025-11-18T15:14:23.117ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2672025-11-18T15:14:23.120ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2682025-11-18T15:14:23.642ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692025-11-18T15:14:23.647ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2702025-11-18T15:14:23.650ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2712025-11-18T15:14:23.829ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15694099324149499291 n_samples = 6 unit = oximeter
2722025-11-18T15:14:24.312ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2732025-11-18T15:14:24.313ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 71.786883262s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
2742025-11-18T15:14:24.999ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2752025-11-18T15:14:25.005ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2762025-11-18T15:14:25.008ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2772025-11-18T15:14:25.365ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3bd16592-14ca-4d00-959b-9efb0f600b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6b993aa7-f2e0-4e9a-b1d1-ea7a8ea320d6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 3861, tv_nsec: 435146317 } }) unit = producer-server
2782025-11-18T15:14:25.365ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2792025-11-18T15:14:25.403ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2802025-11-18T15:14:25.705ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2812025-11-18T15:14:25.712ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2822025-11-18T15:14:25.717ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2832025-11-18T15:14:26.781ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2842025-11-18T15:14:26.790ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2852025-11-18T15:14:26.794ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2862025-11-18T15:14:28.220ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2872025-11-18T15:14:28.226ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2882025-11-18T15:14:28.230ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2892025-11-18T15:14:29.283ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2902025-11-18T15:14:29.288ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2912025-11-18T15:14:29.297ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2922025-11-18T15:14:30.492ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2932025-11-18T15:14:30.500ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2942025-11-18T15:14:30.503ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2952025-11-18T15:14:31.169ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2962025-11-18T15:14:31.178ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
2972025-11-18T15:14:31.182ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2982025-11-18T15:14:32.499ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2992025-11-18T15:14:32.504ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3002025-11-18T15:14:32.509ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3012025-11-18T15:14:32.654ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter
3022025-11-18T15:14:33.617ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3032025-11-18T15:14:33.631ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3042025-11-18T15:14:33.636ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3052025-11-18T15:14:33.833ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15694099324149499291 n_samples = 6 unit = oximeter
3062025-11-18T15:14:34.630ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3072025-11-18T15:14:34.639ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3082025-11-18T15:14:34.646ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3092025-11-18T15:14:36.132ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3102025-11-18T15:14:36.140ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3112025-11-18T15:14:36.143ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3122025-11-18T15:14:37.552ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3132025-11-18T15:14:37.558ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3142025-11-18T15:14:37.561ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3152025-11-18T15:14:38.632ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3162025-11-18T15:14:38.638ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3172025-11-18T15:14:38.641ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3182025-11-18T15:14:39.721ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3192025-11-18T15:14:39.726ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3202025-11-18T15:14:39.730ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3212025-11-18T15:14:40.404ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3222025-11-18T15:14:40.405ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 87.878305334s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/e76962a/oximeter/producer/src/lib.rs:371 unit = producer-server
3232025-11-18T15:14:40.866ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3242025-11-18T15:14:40.871ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3252025-11-18T15:14:40.875ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3262025-11-18T15:14:42.160ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3272025-11-18T15:14:42.166ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3282025-11-18T15:14:42.170ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3292025-11-18T15:14:42.657ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter
3302025-11-18T15:14:42.904ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3312025-11-18T15:14:42.911ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3322025-11-18T15:14:42.914ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3332025-11-18T15:14:43.837ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15694099324149499291 n_samples = 6 unit = oximeter
3342025-11-18T15:14:44.322ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3352025-11-18T15:14:44.328ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3362025-11-18T15:14:44.331ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3372025-11-18T15:14:44.853ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382025-11-18T15:14:44.858ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3392025-11-18T15:14:44.862ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3402025-11-18T15:14:45.212ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3bd16592-14ca-4d00-959b-9efb0f600b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6b993aa7-f2e0-4e9a-b1d1-ea7a8ea320d6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ca18d5c5-ecce-41d0-8dd0-07e6b45d67ac.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 3881, tv_nsec: 282340585 } }) unit = producer-server
3412025-11-18T15:14:45.213ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
3422025-11-18T15:14:45.246ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3432025-11-18T15:14:45.261ZDEBGtfportd (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "04972f16-beb1-4eb6-8a96-77da99f17bad", "content-length": "41", "date": "Tue, 18 Nov 2025 15:14:45 GMT"} }) unit = producer-server
3442025-11-18T15:14:45.264ZDEBGtfportd (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s unit = producer-server wait_period = 150s
3452025-11-18T15:14:46.280ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3462025-11-18T15:14:46.286ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3472025-11-18T15:14:46.289ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3482025-11-18T15:14:47.422ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3492025-11-18T15:14:47.428ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3502025-11-18T15:14:47.432ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3512025-11-18T15:14:48.358ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3522025-11-18T15:14:48.363ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3532025-11-18T15:14:48.366ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3542025-11-18T15:14:49.270ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3552025-11-18T15:14:49.276ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3562025-11-18T15:14:49.280ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3572025-11-18T15:14:50.398ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3582025-11-18T15:14:50.403ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3592025-11-18T15:14:50.406ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3602025-11-18T15:14:50.958ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3612025-11-18T15:14:50.964ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3622025-11-18T15:14:50.968ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3632025-11-18T15:14:52.062ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3642025-11-18T15:14:52.068ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3652025-11-18T15:14:52.071ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3662025-11-18T15:14:52.660ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter
3672025-11-18T15:14:53.279ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3682025-11-18T15:14:53.286ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3692025-11-18T15:14:53.289ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3702025-11-18T15:14:53.839ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15694099324149499291 n_samples = 6 unit = oximeter
3712025-11-18T15:14:54.510ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3722025-11-18T15:14:54.516ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3732025-11-18T15:14:54.519ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3742025-11-18T15:14:56.018ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3752025-11-18T15:14:56.025ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3762025-11-18T15:14:56.029ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3772025-11-18T15:14:57.444ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3782025-11-18T15:14:57.451ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3792025-11-18T15:14:57.454ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3802025-11-18T15:14:58.629ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3812025-11-18T15:14:58.635ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3822025-11-18T15:14:58.639ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3832025-11-18T15:15:00.105ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3842025-11-18T15:15:00.111ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3852025-11-18T15:15:00.115ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3862025-11-18T15:15:01.047ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3872025-11-18T15:15:01.056ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3882025-11-18T15:15:01.060ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3892025-11-18T15:15:02.346ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3902025-11-18T15:15:02.356ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::cc33:bfff:fe58:fe04]
3912025-11-18T15:15:02.360ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3922025-11-18T15:15:02.663ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 119716041416039592 n_samples = 6 unit = oximeter