Max level shown:
1 [ Mar 3 23:29:21 Enabled. ]
2 [ Mar 3 23:29:21 Rereading configuration. ]
3 [ Mar 3 23:29:22 Rereading configuration. ]
4 [ Mar 3 23:29:34 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Mar 3 23:29:34 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72025-03-03T23:29:34.254ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82025-03-03T23:29:34.317ZINFOtfportd: connected to dpd running softnpu-b37a8fdc-dirty
92025-03-03T23:29:34.317ZINFOtfportd: running in sync-only mode
102025-03-03T23:29:34.317ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112025-03-03T23:29:34.327ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132025-03-03T23:29:34.328ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 9da135d7-c597-4ef0-a20e-3484e6fa33ee, ), sled_id: Some( c208ba3e-49ee-460e-bbc7-267cd04132b6, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142025-03-03T23:29:34.328ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152025-03-03T23:29:34.332ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
162025-03-03T23:29:34.336ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::9c5d:27ff:fefc:e58d->9e:5d:27:fc:e5:8d interface = tfportrear0_0
172025-03-03T23:29:34.336ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::d836:52ff:fe84:87bf->da:36:52:84:87:bf interface = tfportrear0_0
182025-03-03T23:29:34.344ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
192025-03-03T23:29:34.351ZWARNtfportd: tfportqsfp0_0 has no addrs
20 [ Mar 3 23:29:34 Rereading configuration. ]
21 [ Mar 3 23:29:34 Executing refresh method (:kill). ]
22 [ Mar 3 23:29:34 Stopping because all processes in service exited. ]
23 [ Mar 3 23:29:34 Executing stop method (:kill). ]
24 [ Mar 3 23:29:34 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
25 [ Mar 3 23:29:34 Method "start" exited with status 0. ]
262025-03-03T23:29:34.892ZINFOtfportd: connecting to dpd host = [::1] port = 12224
272025-03-03T23:29:34.932ZINFOtfportd: connected to dpd running softnpu-b37a8fdc-dirty
282025-03-03T23:29:34.932ZINFOtfportd: running in sync-only mode
292025-03-03T23:29:34.932ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
302025-03-03T23:29:34.936ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
312025-03-03T23:29:34.940ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 9da135d7-c597-4ef0-a20e-3484e6fa33ee, sled_id: c208ba3e-49ee-460e-bbc7-267cd04132b6, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, lot: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: e258169b-98c1-4cdb-806b-c127ab718977, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
322025-03-03T23:29:34.940ZDEBGtfportd: created producer ID producer_id = bdf19972-14b8-4425-99a2-d3df9dca882f unit = oximeter
332025-03-03T23:29:34.946ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 9da135d7-c597-4ef0-a20e-3484e6fa33ee, ), sled_id: Some( c208ba3e-49ee-460e-bbc7-267cd04132b6, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
342025-03-03T23:29:34.949ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::9c5d:27ff:fefc:e58d->9e:5d:27:fc:e5:8d interface = tfportrear0_0
352025-03-03T23:29:34.949ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::d836:52ff:fe84:87bf->da:36:52:84:87:bf interface = tfportrear0_0
362025-03-03T23:29:34.952ZDEBGtfportd: registered DTrace probes unit = producer-server
372025-03-03T23:29:34.953ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
382025-03-03T23:29:34.956ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:40229 method = GET path = /{producer_id} unit = producer-server versions = all
392025-03-03T23:29:34.956ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.15.1/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:40229 unit = producer-server
402025-03-03T23:29:34.956ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:40229 unit = producer-server
412025-03-03T23:29:34.956ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:40229 unit = producer-server
422025-03-03T23:29:34.956ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
432025-03-03T23:29:34.956ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/78772ea/internal-dns/resolver/src/resolver.rs:111 unit = producer-server
442025-03-03T23:29:34.957ZDEBGtfportd: starting producer registration task unit = producer-server
452025-03-03T23:29:34.957ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:40229 file = /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/78772ea/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = bdf19972-14b8-4425-99a2-d3df9dca882f unit = producer-server
462025-03-03T23:29:34.957ZINFOtfportd: created oximeter producer server unit = oximeter
472025-03-03T23:29:34.957ZDEBGtfportd (producer-registration-task): registering / renewing oximeter producer lease with Nexus unit = producer-server
482025-03-03T23:29:34.965ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
492025-03-03T23:29:34.968ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2025-03-03T23:29:17.652824590Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502025-03-03T23:29:34.969ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 464913295336443657 unit = oximeter
512025-03-03T23:29:34.969ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522025-03-03T23:29:34.971ZWARNtfportd: tfportqsfp0_0 has no addrs
532025-03-03T23:29:36.047ZINFOtfportd: added new ARP entry entry = 10.151.1.174->82:1b:ea:fe:7f:9c interface = tfportqsfp0_0
542025-03-03T23:29:36.382ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
552025-03-03T23:29:36.392ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
562025-03-03T23:29:36.397ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
572025-03-03T23:29:36.398ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2025-03-03T23:29:17.652688027Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
582025-03-03T23:29:36.398ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 13951411851100592250 unit = oximeter
592025-03-03T23:29:36.398ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
602025-03-03T23:29:37.059ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
612025-03-03T23:29:37.069ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
622025-03-03T23:29:37.073ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
632025-03-03T23:29:38.161ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
642025-03-03T23:29:38.171ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
652025-03-03T23:29:38.175ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
662025-03-03T23:29:39.461ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
672025-03-03T23:29:39.474ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
682025-03-03T23:29:39.480ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
692025-03-03T23:29:40.232ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
702025-03-03T23:29:40.243ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
712025-03-03T23:29:40.248ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
722025-03-03T23:29:41.334ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
732025-03-03T23:29:41.345ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
742025-03-03T23:29:41.351ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
752025-03-03T23:29:41.938ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
762025-03-03T23:29:41.948ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
772025-03-03T23:29:41.953ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
782025-03-03T23:29:43.267ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
792025-03-03T23:29:43.277ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
802025-03-03T23:29:43.282ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
812025-03-03T23:29:44.263ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
822025-03-03T23:29:44.272ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
832025-03-03T23:29:44.277ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
842025-03-03T23:29:44.961ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3d39a650-6466-497b-a5de-b0f59c63af58.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("4941fee5-459f-4a53-8c4c-fbdde4f796ae.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("51fdb59d-a0fa-471e-a35a-8891e1620d32.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("51fdb59d-a0fa-471e-a35a-8891e1620d32.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::b))) }], valid_until: Instant { tv_sec: 853, tv_nsec: 809183230 } }) unit = producer-server
852025-03-03T23:29:44.961ZDEBGtfportd (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::a]:12221 unit = producer-server
862025-03-03T23:29:44.994ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
872025-03-03T23:29:44.999ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 464913295336443657 n_samples = 6 unit = oximeter
882025-03-03T23:29:45.005ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
892025-03-03T23:29:45.016ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
902025-03-03T23:29:45.021ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
912025-03-03T23:29:46.122ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
922025-03-03T23:29:46.134ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
932025-03-03T23:29:46.140ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
942025-03-03T23:29:46.400ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13951411851100592250 n_samples = 6 unit = oximeter
952025-03-03T23:29:47.301ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
962025-03-03T23:29:47.312ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
972025-03-03T23:29:47.317ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
982025-03-03T23:29:48.632ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
992025-03-03T23:29:48.643ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1002025-03-03T23:29:48.648ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1012025-03-03T23:29:49.412ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1022025-03-03T23:29:49.422ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1032025-03-03T23:29:49.427ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1042025-03-03T23:29:50.300ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1052025-03-03T23:29:50.310ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1062025-03-03T23:29:50.315ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1072025-03-03T23:29:51.560ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1082025-03-03T23:29:51.570ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1092025-03-03T23:29:51.575ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1102025-03-03T23:29:52.857ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1112025-03-03T23:29:52.866ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1122025-03-03T23:29:52.871ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1132025-03-03T23:29:53.447ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1142025-03-03T23:29:53.457ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1152025-03-03T23:29:53.462ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1162025-03-03T23:29:54.090ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1172025-03-03T23:29:54.101ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1182025-03-03T23:29:54.106ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1192025-03-03T23:29:54.930ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1202025-03-03T23:29:54.940ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1212025-03-03T23:29:54.945ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1222025-03-03T23:29:55.005ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 464913295336443657 n_samples = 6 unit = oximeter
1232025-03-03T23:29:56.221ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1242025-03-03T23:29:56.232ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1252025-03-03T23:29:56.237ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1262025-03-03T23:29:56.410ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13951411851100592250 n_samples = 6 unit = oximeter
1272025-03-03T23:29:57.741ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282025-03-03T23:29:57.751ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1292025-03-03T23:29:57.756ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1302025-03-03T23:29:58.530ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312025-03-03T23:29:58.540ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1322025-03-03T23:29:58.545ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1332025-03-03T23:29:59.640ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342025-03-03T23:29:59.651ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1352025-03-03T23:29:59.657ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1362025-03-03T23:30:00.011ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1372025-03-03T23:30:00.012ZWARNtfportd (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 183.996872ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/78772ea/oximeter/producer/src/lib.rs:420 unit = producer-server
1382025-03-03T23:30:00.281ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1392025-03-03T23:30:00.380ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402025-03-03T23:30:00.524ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c5d:27ff:fefc:e58d]
1412025-03-03T23:30:00.541ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]