Max level shown:
1 [ Feb 25 04:29:20 Enabled. ]
2 [ Feb 25 04:29:20 Rereading configuration. ]
3 [ Feb 25 04:29:20 Rereading configuration. ]
4 [ Feb 25 04:29:32 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Feb 25 04:29:32 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72025-02-25T04:29:32.626ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82025-02-25T04:29:32.691ZINFOtfportd: connected to dpd running softnpu-b37a8fdc-dirty
92025-02-25T04:29:32.691ZINFOtfportd: running in sync-only mode
102025-02-25T04:29:32.691ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112025-02-25T04:29:32.698ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132025-02-25T04:29:32.702ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 14ed7c59-d273-4c87-9760-2414ab7e6fbd, ), sled_id: Some( 3b5b8c63-3f71-4c55-9fbd-e4ff49164727, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142025-02-25T04:29:32.702ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152025-02-25T04:29:32.702ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::a436:f9ff:fe4f:fc86->a6:36:f9:4f:fc:86 interface = tfportrear0_0
162025-02-25T04:29:32.703ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::3877:46ff:fe91:1c9->3a:77:46:91:01:c9 interface = tfportrear0_0
172025-02-25T04:29:32.707ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
182025-02-25T04:29:32.719ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
192025-02-25T04:29:32.727ZWARNtfportd: tfportqsfp0_0 has no addrs
20 [ Feb 25 04:29:33 Rereading configuration. ]
21 [ Feb 25 04:29:33 Executing refresh method (:kill). ]
22 [ Feb 25 04:29:33 Stopping because all processes in service exited. ]
23 [ Feb 25 04:29:33 Executing stop method (:kill). ]
24 [ Feb 25 04:29:33 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
25 [ Feb 25 04:29:33 Method "start" exited with status 0. ]
262025-02-25T04:29:33.243ZINFOtfportd: connecting to dpd host = [::1] port = 12224
272025-02-25T04:29:33.284ZINFOtfportd: connected to dpd running softnpu-b37a8fdc-dirty
282025-02-25T04:29:33.284ZINFOtfportd: running in sync-only mode
292025-02-25T04:29:33.284ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
302025-02-25T04:29:33.291ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
312025-02-25T04:29:33.294ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 14ed7c59-d273-4c87-9760-2414ab7e6fbd, sled_id: 3b5b8c63-3f71-4c55-9fbd-e4ff49164727, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, lot: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: b1bb074b-7706-49e5-b4a4-766c5ab1bd39, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
322025-02-25T04:29:33.294ZDEBGtfportd: created producer ID producer_id = 9e0df771-9a04-4937-bc79-b687b8d995a3 unit = oximeter
332025-02-25T04:29:33.296ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 14ed7c59-d273-4c87-9760-2414ab7e6fbd, ), sled_id: Some( 3b5b8c63-3f71-4c55-9fbd-e4ff49164727, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
342025-02-25T04:29:33.300ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::a436:f9ff:fe4f:fc86->a6:36:f9:4f:fc:86 interface = tfportrear0_0
352025-02-25T04:29:33.301ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::3877:46ff:fe91:1c9->3a:77:46:91:01:c9 interface = tfportrear0_0
362025-02-25T04:29:33.301ZDEBGtfportd: registered DTrace probes unit = producer-server
372025-02-25T04:29:33.302ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:38032 method = GET path = /{producer_id} unit = producer-server versions = all
382025-02-25T04:29:33.302ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.15.1/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:38032 unit = producer-server
392025-02-25T04:29:33.302ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:38032 unit = producer-server
402025-02-25T04:29:33.302ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:38032 unit = producer-server
412025-02-25T04:29:33.302ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422025-02-25T04:29:33.302ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/78772ea/internal-dns/resolver/src/resolver.rs:111 unit = producer-server
432025-02-25T04:29:33.302ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
442025-02-25T04:29:33.302ZDEBGtfportd: starting producer registration task unit = producer-server
452025-02-25T04:29:33.305ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:38032 file = /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/78772ea/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 9e0df771-9a04-4937-bc79-b687b8d995a3 unit = producer-server
462025-02-25T04:29:33.305ZINFOtfportd: created oximeter producer server unit = oximeter
472025-02-25T04:29:33.305ZDEBGtfportd (producer-registration-task): registering / renewing oximeter producer lease with Nexus unit = producer-server
482025-02-25T04:29:33.315ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
492025-02-25T04:29:33.320ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2025-02-25T04:29:15.755192542Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502025-02-25T04:29:33.320ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 13044355543825491914 unit = oximeter
512025-02-25T04:29:33.320ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522025-02-25T04:29:33.322ZWARNtfportd: tfportqsfp0_0 has no addrs
532025-02-25T04:29:34.364ZINFOtfportd: added new ARP entry entry = 10.151.1.174->b2:0d:f4:3b:9f:76 interface = tfportqsfp0_0
542025-02-25T04:29:34.428ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
552025-02-25T04:29:34.438ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
562025-02-25T04:29:34.442ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
572025-02-25T04:29:34.443ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2025-02-25T04:29:15.755053019Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
582025-02-25T04:29:34.443ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 16831656795078331904 unit = oximeter
592025-02-25T04:29:34.443ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
602025-02-25T04:29:35.324ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
612025-02-25T04:29:35.335ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
622025-02-25T04:29:35.340ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
632025-02-25T04:29:36.457ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
642025-02-25T04:29:36.468ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
652025-02-25T04:29:36.474ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
662025-02-25T04:29:37.934ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
672025-02-25T04:29:37.945ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
682025-02-25T04:29:37.950ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
692025-02-25T04:29:38.723ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
702025-02-25T04:29:38.734ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
712025-02-25T04:29:38.739ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
722025-02-25T04:29:39.641ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
732025-02-25T04:29:39.651ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
742025-02-25T04:29:39.656ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
752025-02-25T04:29:41.162ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
762025-02-25T04:29:41.173ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
772025-02-25T04:29:41.178ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
782025-02-25T04:29:42.472ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
792025-02-25T04:29:42.482ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
802025-02-25T04:29:42.487ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
812025-02-25T04:29:43.310ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("0574aa44-c442-4e92-a74d-634e5dc20afa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a607c9bc-1542-483e-9a3e-734a6cdd9259.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("bd747fc9-c33a-48e0-ac15-e5cedcf13dd9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("bd747fc9-c33a-48e0-ac15-e5cedcf13dd9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::a))) }], valid_until: Instant { tv_sec: 597, tv_nsec: 889681900 } }) unit = producer-server
822025-02-25T04:29:43.311ZDEBGtfportd (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221 unit = producer-server
832025-02-25T04:29:43.344ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
842025-02-25T04:29:43.347ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13044355543825491914 n_samples = 6 unit = oximeter
852025-02-25T04:29:43.945ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862025-02-25T04:29:43.956ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
872025-02-25T04:29:43.961ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
882025-02-25T04:29:44.447ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16831656795078331904 n_samples = 6 unit = oximeter
892025-02-25T04:29:44.859ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902025-02-25T04:29:44.871ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
912025-02-25T04:29:44.877ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
922025-02-25T04:29:46.287ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
932025-02-25T04:29:46.298ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
942025-02-25T04:29:46.303ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
952025-02-25T04:29:46.876ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
962025-02-25T04:29:46.886ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
972025-02-25T04:29:46.892ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
982025-02-25T04:29:47.742ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
992025-02-25T04:29:47.752ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1002025-02-25T04:29:47.757ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1012025-02-25T04:29:48.875ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1022025-02-25T04:29:48.885ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1032025-02-25T04:29:48.890ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1042025-02-25T04:29:49.532ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1052025-02-25T04:29:49.543ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1062025-02-25T04:29:49.549ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1072025-02-25T04:29:50.318ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1082025-02-25T04:29:50.328ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1092025-02-25T04:29:50.333ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1102025-02-25T04:29:51.098ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1112025-02-25T04:29:51.109ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1122025-02-25T04:29:51.115ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1132025-02-25T04:29:51.905ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1142025-02-25T04:29:51.915ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1152025-02-25T04:29:51.920ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1162025-02-25T04:29:52.682ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1172025-02-25T04:29:52.692ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1182025-02-25T04:29:52.697ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1192025-02-25T04:29:53.350ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13044355543825491914 n_samples = 6 unit = oximeter
1202025-02-25T04:29:54.059ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1212025-02-25T04:29:54.069ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1222025-02-25T04:29:54.074ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1232025-02-25T04:29:54.449ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16831656795078331904 n_samples = 6 unit = oximeter
1242025-02-25T04:29:55.156ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1252025-02-25T04:29:55.166ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1262025-02-25T04:29:55.171ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1272025-02-25T04:29:55.696ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282025-02-25T04:29:55.706ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1292025-02-25T04:29:55.711ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1302025-02-25T04:29:56.976ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312025-02-25T04:29:56.985ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1322025-02-25T04:29:56.990ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1332025-02-25T04:29:57.767ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342025-02-25T04:29:57.777ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1352025-02-25T04:29:57.782ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1362025-02-25T04:29:58.346ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1372025-02-25T04:29:58.347ZWARNtfportd (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 134.644868ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/78772ea/oximeter/producer/src/lib.rs:420 unit = producer-server
1382025-02-25T04:29:58.514ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1392025-02-25T04:29:58.934ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402025-02-25T04:29:58.944ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1412025-02-25T04:29:58.948ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1422025-02-25T04:29:59.711ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432025-02-25T04:29:59.722ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::3877:46ff:fe91:1c9]
1442025-02-25T04:29:59.727ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1452025-02-25T04:30:00.675ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]