Max level shown:
1 [ Apr 14 23:28:42 Enabled. ]
2 [ Apr 14 23:28:42 Rereading configuration. ]
3 [ Apr 14 23:28:43 Rereading configuration. ]
4 [ Apr 14 23:28:54 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Apr 14 23:28:54 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-04-14T23:28:54.479ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-04-14T23:28:54.481ZINFOtfportd: connected to dpd running softnpu-44a949c9
92026-04-14T23:28:54.481ZINFOtfportd: running in sync-only mode
102026-04-14T23:28:54.482ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-04-14T23:28:54.484ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-04-14T23:28:54.485ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 4c3b02a3-15d6-4701-885e-d417b12f060b, ), sled_id: Some( 13d12140-d02f-4ba5-aaf9-a0b5f293949b, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
142026-04-14T23:28:54.485ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-04-14T23:28:54.490ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::10b7:60ff:fe92:b913->12:b7:60:92:b9:13 interface = tfportrear0_0
162026-04-14T23:28:54.491ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-04-14T23:28:54.495ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
182026-04-14T23:28:54.500ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Apr 14 23:28:55 Rereading configuration. ]
20 [ Apr 14 23:28:55 Executing refresh method (:kill). ]
21 [ Apr 14 23:28:55 Stopping because all processes in service exited. ]
22 [ Apr 14 23:28:55 Executing stop method (:kill). ]
23 [ Apr 14 23:28:55 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Apr 14 23:28:55 Method "start" exited with status 0. ]
252026-04-14T23:28:55.066ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-04-14T23:28:55.067ZINFOtfportd: connected to dpd running softnpu-44a949c9
272026-04-14T23:28:55.068ZINFOtfportd: running in sync-only mode
282026-04-14T23:28:55.068ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-04-14T23:28:55.070ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-04-14T23:28:55.070ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 4c3b02a3-15d6-4701-885e-d417b12f060b, sled_id: 13d12140-d02f-4ba5-aaf9-a0b5f293949b, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 92cba5fe-ea8b-4b19-9430-38aa3f8c6038, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-04-14T23:28:55.070ZDEBGtfportd: created producer ID producer_id = cd21861d-2f72-47bd-b697-4c087c30f2a8 unit = oximeter
322026-04-14T23:28:55.071ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 4c3b02a3-15d6-4701-885e-d417b12f060b, ), sled_id: Some( 13d12140-d02f-4ba5-aaf9-a0b5f293949b, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
332026-04-14T23:28:55.078ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-04-14T23:28:55.079ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::10b7:60ff:fe92:b913->12:b7:60:92:b9:13 interface = tfportrear0_0
352026-04-14T23:28:55.079ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::540a:d2ff:fe30:54b3->56:0a:d2:30:54:b3 interface = tfportrear0_0
362026-04-14T23:28:55.082ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-04-14T23:28:55.082ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:45358 method = GET path = /{producer_id} unit = producer-server versions = all
382026-04-14T23:28:55.082ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:45358 unit = producer-server
392026-04-14T23:28:55.083ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:45358 unit = producer-server
402026-04-14T23:28:55.083ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:45358 unit = producer-server
412026-04-14T23:28:55.083ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-04-14T23:28:55.083ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-04-14T23:28:55.083ZDEBGtfportd: starting producer registration task unit = producer-server
442026-04-14T23:28:55.083ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
452026-04-14T23:28:55.083ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:45358 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = cd21861d-2f72-47bd-b697-4c087c30f2a8 unit = producer-server
462026-04-14T23:28:55.083ZINFOtfportd: created oximeter producer server unit = oximeter
472026-04-14T23:28:55.083ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
482026-04-14T23:28:55.084ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-04-14T23:28:55.085ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-04-14T23:28:38.491155587Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502026-04-14T23:28:55.085ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 16381532087394542488 unit = oximeter
512026-04-14T23:28:55.085ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522026-04-14T23:28:55.086ZWARNtfportd: tfportqsfp0_0 has no addrs
532026-04-14T23:28:55.771ZINFOtfportd: added new ARP entry entry = 10.151.2.174->4a:30:b2:74:c9:0f interface = tfportqsfp0_0
542026-04-14T23:28:56.198ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
552026-04-14T23:28:56.203ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
562026-04-14T23:28:56.206ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
572026-04-14T23:28:56.206ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
582026-04-14T23:28:56.207ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-04-14T23:28:38.491024153Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
592026-04-14T23:28:56.207ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 1073861743392637374 unit = oximeter
602026-04-14T23:28:56.207ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
612026-04-14T23:28:57.664ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-04-14T23:28:57.669ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
632026-04-14T23:28:57.672ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
642026-04-14T23:28:58.516ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-04-14T23:28:58.522ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
662026-04-14T23:28:58.526ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
672026-04-14T23:28:59.603ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-04-14T23:28:59.607ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
692026-04-14T23:28:59.610ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
702026-04-14T23:29:00.087ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("17c8edf0-363b-4df3-9b76-00a6819a6c13.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41f1d75d-c504-452d-b8ba-a03e78dfb406.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d914c97d-c0b5-40fb-ba7e-f8f68aec9385.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d914c97d-c0b5-40fb-ba7e-f8f68aec9385.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8880, tv_nsec: 419979340 } }) unit = producer-server
712026-04-14T23:29:00.087ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
722026-04-14T23:29:00.130ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
732026-04-14T23:29:00.845ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-04-14T23:29:00.850ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
752026-04-14T23:29:00.853ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
762026-04-14T23:29:02.025ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-04-14T23:29:02.030ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
782026-04-14T23:29:02.033ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
792026-04-14T23:29:02.900ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-04-14T23:29:02.905ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
812026-04-14T23:29:02.907ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
822026-04-14T23:29:04.107ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-04-14T23:29:04.111ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
842026-04-14T23:29:04.114ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
852026-04-14T23:29:04.635ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862026-04-14T23:29:04.639ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
872026-04-14T23:29:04.642ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
882026-04-14T23:29:05.090ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16381532087394542488 n_samples = 6 unit = oximeter
892026-04-14T23:29:05.500ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902026-04-14T23:29:05.505ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
912026-04-14T23:29:05.508ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
922026-04-14T23:29:06.210ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1073861743392637374 n_samples = 6 unit = oximeter
932026-04-14T23:29:06.808ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942026-04-14T23:29:06.812ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
952026-04-14T23:29:06.816ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
962026-04-14T23:29:07.573ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-04-14T23:29:07.577ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
982026-04-14T23:29:07.580ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
992026-04-14T23:29:08.748ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002026-04-14T23:29:08.753ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1012026-04-14T23:29:08.756ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1022026-04-14T23:29:09.412ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-04-14T23:29:09.417ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1042026-04-14T23:29:09.420ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1052026-04-14T23:29:10.773ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-04-14T23:29:10.778ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1072026-04-14T23:29:10.781ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1082026-04-14T23:29:12.178ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-04-14T23:29:12.182ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1102026-04-14T23:29:12.186ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1112026-04-14T23:29:13.246ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-04-14T23:29:13.250ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1132026-04-14T23:29:13.253ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1142026-04-14T23:29:14.533ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-04-14T23:29:14.538ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1162026-04-14T23:29:14.541ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1172026-04-14T23:29:15.097ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16381532087394542488 n_samples = 6 unit = oximeter
1182026-04-14T23:29:15.135ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1192026-04-14T23:29:15.136ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 20.050009198s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1202026-04-14T23:29:15.337ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("17c8edf0-363b-4df3-9b76-00a6819a6c13.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41f1d75d-c504-452d-b8ba-a03e78dfb406.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d914c97d-c0b5-40fb-ba7e-f8f68aec9385.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d914c97d-c0b5-40fb-ba7e-f8f68aec9385.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8895, tv_nsec: 666297971 } }) unit = producer-server
1212026-04-14T23:29:15.337ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1222026-04-14T23:29:15.370ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1232026-04-14T23:29:15.497ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1242026-04-14T23:29:15.502ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1252026-04-14T23:29:15.505ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1262026-04-14T23:29:16.219ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1073861743392637374 n_samples = 6 unit = oximeter
1272026-04-14T23:29:16.467ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282026-04-14T23:29:16.472ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1292026-04-14T23:29:16.475ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1302026-04-14T23:29:18.258ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312026-04-14T23:29:18.269ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1322026-04-14T23:29:18.275ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1332026-04-14T23:29:18.843ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342026-04-14T23:29:18.850ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1352026-04-14T23:29:18.854ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1362026-04-14T23:29:19.522ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-04-14T23:29:19.529ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1382026-04-14T23:29:19.532ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1392026-04-14T23:29:20.469ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-04-14T23:29:20.474ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1412026-04-14T23:29:20.478ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1422026-04-14T23:29:21.253ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-04-14T23:29:21.259ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1442026-04-14T23:29:21.262ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1452026-04-14T23:29:22.676ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-04-14T23:29:22.681ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1472026-04-14T23:29:22.684ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1482026-04-14T23:29:23.239ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-04-14T23:29:23.244ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1502026-04-14T23:29:23.247ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1512026-04-14T23:29:24.140ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-04-14T23:29:24.146ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1532026-04-14T23:29:24.149ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1542026-04-14T23:29:25.017ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1552026-04-14T23:29:25.022ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1562026-04-14T23:29:25.025ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1572026-04-14T23:29:25.121ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16381532087394542488 n_samples = 6 unit = oximeter
1582026-04-14T23:29:25.996ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1592026-04-14T23:29:26.001ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1602026-04-14T23:29:26.004ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1612026-04-14T23:29:26.239ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1073861743392637374 n_samples = 6 unit = oximeter
1622026-04-14T23:29:27.126ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1632026-04-14T23:29:27.131ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1642026-04-14T23:29:27.134ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1652026-04-14T23:29:28.085ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1662026-04-14T23:29:28.091ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1672026-04-14T23:29:28.095ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1682026-04-14T23:29:28.941ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1692026-04-14T23:29:28.946ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1702026-04-14T23:29:28.949ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1712026-04-14T23:29:30.144ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-04-14T23:29:30.149ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1732026-04-14T23:29:30.151ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1742026-04-14T23:29:30.394ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1752026-04-14T23:29:30.395ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 35.286942655s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1762026-04-14T23:29:30.722ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1772026-04-14T23:29:30.727ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1782026-04-14T23:29:30.730ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1792026-04-14T23:29:30.942ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("17c8edf0-363b-4df3-9b76-00a6819a6c13.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41f1d75d-c504-452d-b8ba-a03e78dfb406.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d914c97d-c0b5-40fb-ba7e-f8f68aec9385.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d914c97d-c0b5-40fb-ba7e-f8f68aec9385.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8911, tv_nsec: 249199883 } }) unit = producer-server
1802026-04-14T23:29:30.942ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1812026-04-14T23:29:30.975ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1822026-04-14T23:29:32.200ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1832026-04-14T23:29:32.205ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1842026-04-14T23:29:32.208ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1852026-04-14T23:29:32.758ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862026-04-14T23:29:32.763ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1872026-04-14T23:29:32.766ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1882026-04-14T23:29:33.857ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-04-14T23:29:33.862ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1902026-04-14T23:29:33.865ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1912026-04-14T23:29:35.126ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16381532087394542488 n_samples = 6 unit = oximeter
1922026-04-14T23:29:35.363ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1932026-04-14T23:29:35.369ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1942026-04-14T23:29:35.372ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1952026-04-14T23:29:36.015ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1962026-04-14T23:29:36.019ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
1972026-04-14T23:29:36.022ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1982026-04-14T23:29:36.242ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1073861743392637374 n_samples = 6 unit = oximeter
1992026-04-14T23:29:36.604ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-04-14T23:29:36.610ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2012026-04-14T23:29:36.613ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2022026-04-14T23:29:37.810ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032026-04-14T23:29:37.813ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2042026-04-14T23:29:37.816ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2052026-04-14T23:29:39.120ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2062026-04-14T23:29:39.125ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2072026-04-14T23:29:39.128ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2082026-04-14T23:29:40.227ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-04-14T23:29:40.232ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2102026-04-14T23:29:40.235ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2112026-04-14T23:29:40.821ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-04-14T23:29:40.825ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2132026-04-14T23:29:40.828ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2142026-04-14T23:29:42.024ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2152026-04-14T23:29:42.030ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2162026-04-14T23:29:42.033ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2172026-04-14T23:29:43.438ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2182026-04-14T23:29:43.443ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2192026-04-14T23:29:43.445ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2202026-04-14T23:29:44.890ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2212026-04-14T23:29:44.896ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2222026-04-14T23:29:44.900ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2232026-04-14T23:29:45.129ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16381532087394542488 n_samples = 6 unit = oximeter
2242026-04-14T23:29:45.529ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2252026-04-14T23:29:45.545ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2262026-04-14T23:29:45.545ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2272026-04-14T23:29:48.227ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2282026-04-14T23:29:48.227ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2292026-04-14T23:29:48.244ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2302026-04-14T23:29:48.262ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2312026-04-14T23:29:48.278ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 53.166428437s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2322026-04-14T23:29:48.290ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1073861743392637374 n_samples = 6 unit = oximeter
2332026-04-14T23:29:49.264ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-04-14T23:29:49.270ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2352026-04-14T23:29:49.276ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2362026-04-14T23:29:49.357ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("17c8edf0-363b-4df3-9b76-00a6819a6c13.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41f1d75d-c504-452d-b8ba-a03e78dfb406.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d914c97d-c0b5-40fb-ba7e-f8f68aec9385.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d914c97d-c0b5-40fb-ba7e-f8f68aec9385.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 8929, tv_nsec: 662793014 } }) unit = producer-server
2372026-04-14T23:29:49.358ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2382026-04-14T23:29:49.408ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2392026-04-14T23:29:50.213ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402026-04-14T23:29:50.219ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2412026-04-14T23:29:50.223ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2422026-04-14T23:29:51.217ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2432026-04-14T23:29:51.223ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2442026-04-14T23:29:51.227ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2452026-04-14T23:29:52.480ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462026-04-14T23:29:52.487ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2472026-04-14T23:29:52.490ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2482026-04-14T23:29:53.004ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2492026-04-14T23:29:53.012ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2502026-04-14T23:29:53.017ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2512026-04-14T23:29:53.715ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2522026-04-14T23:29:53.723ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2532026-04-14T23:29:53.727ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2542026-04-14T23:29:54.880ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2552026-04-14T23:29:54.888ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2562026-04-14T23:29:54.892ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2572026-04-14T23:29:55.134ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16381532087394542488 n_samples = 6 unit = oximeter
2582026-04-14T23:29:55.660ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2592026-04-14T23:29:55.666ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2602026-04-14T23:29:55.669ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2612026-04-14T23:29:56.756ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2622026-04-14T23:29:56.762ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2632026-04-14T23:29:56.765ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2642026-04-14T23:29:58.189ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2652026-04-14T23:29:58.195ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2662026-04-14T23:29:58.199ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2672026-04-14T23:29:58.294ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1073861743392637374 n_samples = 6 unit = oximeter
2682026-04-14T23:29:59.417ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692026-04-14T23:29:59.427ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2702026-04-14T23:29:59.433ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2712026-04-14T23:30:00.440ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722026-04-14T23:30:00.447ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::10b7:60ff:fe92:b913]
2732026-04-14T23:30:00.450ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]