Max level shown:
1 [ Oct 8 10:40:13 Enabled. ]
2 [ Oct 8 10:40:13 Rereading configuration. ]
3 [ Oct 8 10:40:14 Rereading configuration. ]
4 [ Oct 8 10:40:36 Rereading configuration. ]
5 [ Oct 8 10:40:44 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
6 [ Oct 8 10:40:44 Method "start" exited with status 0. ]
72026-10-08T10:40:44.494ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-10-08T10:40:44.496ZERROtfportd: Failed to connect to dpd. Retrying...
92026-10-08T10:40:45.497ZINFOtfportd: connected to dpd running softnpu-a133c73e
102026-10-08T10:40:45.498ZINFOtfportd: running in sync-only mode
112026-10-08T10:40:45.498ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
122026-10-08T10:40:45.498ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
132026-10-08T10:40:45.499ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 616d2937-8154-40cc-b8af-c334306bfbb2, sled_id: 1df7f81c-8838-4a6c-b531-bdb416622a7c, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: de36ba89-f4e4-45b6-a4e0-a3eae6e00833, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
142026-10-08T10:40:45.499ZDEBGtfportd: created producer ID producer_id = a2ac6ba2-fd88-4f16-a97e-dd6ac0e9d9fc unit = oximeter
152026-10-08T10:40:45.500ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 616d2937-8154-40cc-b8af-c334306bfbb2, ), sled_id: Some( 1df7f81c-8838-4a6c-b531-bdb416622a7c, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
162026-10-08T10:40:45.507ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-10-08T10:40:45.509ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::581a:5aff:fe8a:87e0->5a:1a:5a:8a:87:e0 interface = tfportrear0_0
182026-10-08T10:40:45.509ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::10be:bbff:feec:11ca->00:00:00:00:00:00 interface = tfportrear0_0
192026-10-08T10:40:45.510ZDEBGtfportd: registered DTrace probes unit = producer-server
202026-10-08T10:40:45.510ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:64284 method = GET path = /{producer_id} unit = producer-server versions = all
212026-10-08T10:40:45.510ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:64284 unit = producer-server
222026-10-08T10:40:45.510ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:64284 unit = producer-server
232026-10-08T10:40:45.510ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:64284 unit = producer-server
242026-10-08T10:40:45.510ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
252026-10-08T10:40:45.511ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
262026-10-08T10:40:45.511ZDEBGtfportd: starting producer registration task unit = producer-server
272026-10-08T10:40:45.511ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:64284 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = a2ac6ba2-fd88-4f16-a97e-dd6ac0e9d9fc unit = producer-server
282026-10-08T10:40:45.511ZINFOtfportd: created oximeter producer server unit = oximeter
292026-10-08T10:40:45.511ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
302026-10-08T10:40:45.512ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
312026-10-08T10:40:45.512ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
322026-10-08T10:40:45.512ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("13de7693-7f46-48fe-aae1-8320be3af888.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("394f11a0-b33a-4632-a76e-dc6ff549a64a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("42a559a2-2fa8-474e-9aad-2812d84ed52c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("42a559a2-2fa8-474e-9aad-2812d84ed52c.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 589, tv_nsec: 165850488 } }) unit = producer-server
332026-10-08T10:40:45.513ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-10-08T10:40:09.538468537Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
342026-10-08T10:40:45.513ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 3241964644110830064 unit = oximeter
352026-10-08T10:40:45.513ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
362026-10-08T10:40:45.513ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
372026-10-08T10:40:45.517ZWARNtfportd: tfportqsfp0_0 has no addrs
382026-10-08T10:40:45.529ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
392026-10-08T10:40:46.650ZINFOtfportd: removed stale NDP entry :fe80::10be:bbff:feec:11ca->00:00:00:00:00:00
402026-10-08T10:40:46.650ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::10be:bbff:feec:11ca->12:be:bb:ec:11:ca interface = tfportrear0_0
412026-10-08T10:40:47.020ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
422026-10-08T10:40:47.025ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
432026-10-08T10:40:47.028ZWARNtfportd: tfportqsfp0_0 has no addrs
442026-10-08T10:40:47.805ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
452026-10-08T10:40:47.810ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
462026-10-08T10:40:47.813ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
472026-10-08T10:40:47.813ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
482026-10-08T10:40:47.814ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-10-08T10:40:09.538334884Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
492026-10-08T10:40:47.814ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 17609927112930267511 unit = oximeter
502026-10-08T10:40:47.814ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
512026-10-08T10:40:48.071ZINFOtfportd: added new ARP entry entry = 10.151.1.174->9a:27:c5:e2:3b:7b interface = tfportqsfp0_0
522026-10-08T10:40:48.375ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
532026-10-08T10:40:48.380ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
542026-10-08T10:40:48.382ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
552026-10-08T10:40:49.415ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
562026-10-08T10:40:49.419ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
572026-10-08T10:40:49.422ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
582026-10-08T10:40:50.930ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
592026-10-08T10:40:50.935ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
602026-10-08T10:40:50.937ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
612026-10-08T10:40:53.386ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-10-08T10:40:53.389ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
632026-10-08T10:40:53.392ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
642026-10-08T10:40:53.723ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-10-08T10:40:53.728ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
662026-10-08T10:40:53.731ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
672026-10-08T10:40:54.314ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-10-08T10:40:54.319ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
692026-10-08T10:40:54.321ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
702026-10-08T10:40:55.739ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-10-08T10:40:55.743ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
722026-10-08T10:40:55.746ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
732026-10-08T10:40:55.939ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3241964644110830064 n_samples = 6 unit = oximeter
742026-10-08T10:40:56.357ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
752026-10-08T10:40:56.362ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
762026-10-08T10:40:56.365ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
772026-10-08T10:40:57.339ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
782026-10-08T10:40:57.343ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
792026-10-08T10:40:57.346ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
802026-10-08T10:40:58.244ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17609927112930267511 n_samples = 6 unit = oximeter
812026-10-08T10:40:58.371ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
822026-10-08T10:40:58.376ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
832026-10-08T10:40:58.379ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
842026-10-08T10:40:59.533ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
852026-10-08T10:40:59.536ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
862026-10-08T10:40:59.539ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
872026-10-08T10:41:00.374ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
882026-10-08T10:41:00.377ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
892026-10-08T10:41:00.380ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
902026-10-08T10:41:00.958ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
912026-10-08T10:41:00.959ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 15.020608926s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
922026-10-08T10:41:01.254ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("13de7693-7f46-48fe-aae1-8320be3af888.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("394f11a0-b33a-4632-a76e-dc6ff549a64a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("42a559a2-2fa8-474e-9aad-2812d84ed52c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("42a559a2-2fa8-474e-9aad-2812d84ed52c.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 604, tv_nsec: 480208207 } }) unit = producer-server
932026-10-08T10:41:01.255ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
942026-10-08T10:41:01.261ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
952026-10-08T10:41:02.330ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
962026-10-08T10:41:02.335ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
972026-10-08T10:41:02.338ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
982026-10-08T10:41:02.955ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
992026-10-08T10:41:02.960ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
1002026-10-08T10:41:02.962ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1012026-10-08T10:41:03.604ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1022026-10-08T10:41:03.608ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
1032026-10-08T10:41:03.611ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1042026-10-08T10:41:05.051ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1052026-10-08T10:41:05.055ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
1062026-10-08T10:41:05.057ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1072026-10-08T10:41:06.495ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1082026-10-08T10:41:06.499ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
1092026-10-08T10:41:06.501ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1102026-10-08T10:41:06.542ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3241964644110830064 n_samples = 6 unit = oximeter
1112026-10-08T10:41:07.368ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-10-08T10:41:07.372ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
1132026-10-08T10:41:07.375ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1142026-10-08T10:41:08.489ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-10-08T10:41:08.494ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
1162026-10-08T10:41:08.497ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1172026-10-08T10:41:08.843ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17609927112930267511 n_samples = 6 unit = oximeter
1182026-10-08T10:41:09.949ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1192026-10-08T10:41:09.953ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
1202026-10-08T10:41:09.956ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1212026-10-08T10:41:10.937ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1222026-10-08T10:41:10.941ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
1232026-10-08T10:41:10.943ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1242026-10-08T10:41:11.480ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1252026-10-08T10:41:11.485ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
1262026-10-08T10:41:11.488ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1272026-10-08T10:41:12.928ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282026-10-08T10:41:12.932ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::581a:5aff:fe8a:87e0]
1292026-10-08T10:41:12.935ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]