Max level shown:
1 [ Sep 4 17:58:53 Enabled. ]
2 [ Sep 4 17:58:53 Rereading configuration. ]
3 [ Sep 4 17:58:53 Rereading configuration. ]
4 [ Sep 4 17:59:04 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Sep 4 17:59:04 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-09-04T17:59:04.900ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-09-04T17:59:04.902ZINFOtfportd: connected to dpd running softnpu-5cb3f22d
92026-09-04T17:59:04.903ZINFOtfportd: running in sync-only mode
102026-09-04T17:59:04.903ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-09-04T17:59:04.905ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-09-04T17:59:04.906ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 80dbc06e-2665-4a88-951b-ede22238ef52, ), sled_id: Some( 022b78e2-b63a-4cbf-b36c-cec32898505e, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-09-04T17:59:04.906ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-09-04T17:59:04.911ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::f463:c0ff:feec:668f->f6:63:c0:ec:66:8f interface = tfportrear0_0
162026-09-04T17:59:04.914ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-09-04T17:59:04.921ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
182026-09-04T17:59:04.928ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Sep 4 17:59:05 Rereading configuration. ]
20 [ Sep 4 17:59:05 Executing refresh method (:kill). ]
21 [ Sep 4 17:59:05 Stopping because all processes in service exited. ]
22 [ Sep 4 17:59:05 Executing stop method (:kill). ]
23 [ Sep 4 17:59:05 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Sep 4 17:59:05 Method "start" exited with status 0. ]
252026-09-04T17:59:05.568ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-09-04T17:59:05.569ZINFOtfportd: connected to dpd running softnpu-5cb3f22d
272026-09-04T17:59:05.569ZINFOtfportd: running in sync-only mode
282026-09-04T17:59:05.569ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-09-04T17:59:05.572ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-09-04T17:59:05.572ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 80dbc06e-2665-4a88-951b-ede22238ef52, sled_id: 022b78e2-b63a-4cbf-b36c-cec32898505e, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: fd7202f4-4ba1-40d0-bf42-be29a2752606, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-09-04T17:59:05.572ZDEBGtfportd: created producer ID producer_id = 7f82ba78-dbfe-46e7-9935-9d08a8d5990a unit = oximeter
322026-09-04T17:59:05.574ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 80dbc06e-2665-4a88-951b-ede22238ef52, ), sled_id: Some( 022b78e2-b63a-4cbf-b36c-cec32898505e, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
332026-09-04T17:59:05.582ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-09-04T17:59:05.584ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::ccd9:1fff:fe55:ad8b->ce:d9:1f:55:ad:8b interface = tfportrear0_0
352026-09-04T17:59:05.584ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::f463:c0ff:feec:668f->f6:63:c0:ec:66:8f interface = tfportrear0_0
362026-09-04T17:59:05.587ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-09-04T17:59:05.588ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:39010 method = GET path = /{producer_id} unit = producer-server versions = all
382026-09-04T17:59:05.588ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:39010 unit = producer-server
392026-09-04T17:59:05.589ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:39010 unit = producer-server
402026-09-04T17:59:05.589ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:39010 unit = producer-server
412026-09-04T17:59:05.589ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-09-04T17:59:05.589ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-09-04T17:59:05.589ZDEBGtfportd: starting producer registration task unit = producer-server
442026-09-04T17:59:05.589ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:39010 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 7f82ba78-dbfe-46e7-9935-9d08a8d5990a unit = producer-server
452026-09-04T17:59:05.589ZINFOtfportd: created oximeter producer server unit = oximeter
462026-09-04T17:59:05.589ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-09-04T17:59:05.589ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
482026-09-04T17:59:05.590ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-09-04T17:59:05.593ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-09-04T17:58:48.759923449Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502026-09-04T17:59:05.593ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 16506387734860978422 unit = oximeter
512026-09-04T17:59:05.593ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522026-09-04T17:59:05.594ZWARNtfportd: tfportqsfp0_0 has no addrs
532026-09-04T17:59:06.222ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
542026-09-04T17:59:06.229ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
552026-09-04T17:59:06.233ZWARNtfportd: tfportqsfp0_0 has no addrs
562026-09-04T17:59:06.917ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
572026-09-04T17:59:06.921ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
582026-09-04T17:59:06.925ZWARNtfportd: tfportqsfp0_0 has no addrs
592026-09-04T17:59:08.169ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
602026-09-04T17:59:08.175ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
612026-09-04T17:59:08.178ZWARNtfportd: tfportqsfp0_0 has no addrs
622026-09-04T17:59:08.723ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
632026-09-04T17:59:08.728ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
642026-09-04T17:59:08.731ZWARNtfportd: tfportqsfp0_0 has no addrs
652026-09-04T17:59:09.409ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
662026-09-04T17:59:09.415ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
672026-09-04T17:59:09.418ZWARNtfportd: tfportqsfp0_0 has no addrs
682026-09-04T17:59:09.948ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
692026-09-04T17:59:09.953ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
702026-09-04T17:59:09.956ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
712026-09-04T17:59:09.956ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
722026-09-04T17:59:09.957ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-09-04T17:58:48.759791039Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
732026-09-04T17:59:09.957ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 798518086241873458 unit = oximeter
742026-09-04T17:59:09.957ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
752026-09-04T17:59:10.521ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
762026-09-04T17:59:10.526ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
772026-09-04T17:59:10.529ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
782026-09-04T17:59:10.540ZINFOtfportd: added new ARP entry entry = 10.151.1.174->f6:e4:fb:a4:10:57 interface = tfportqsfp0_0
792026-09-04T17:59:10.593ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("34c992a0-d2ce-4be5-b32a-57ee6226b298.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9380dd25-afd7-42dc-b652-d3dd3b4da0a2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ba8fe925-d548-45b6-922d-69ed5d6bc63d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ba8fe925-d548-45b6-922d-69ed5d6bc63d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 758, tv_nsec: 422947860 } }) unit = producer-server
802026-09-04T17:59:10.593ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
812026-09-04T17:59:10.611ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
822026-09-04T17:59:11.041ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-09-04T17:59:11.046ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
842026-09-04T17:59:11.049ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
852026-09-04T17:59:11.928ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862026-09-04T17:59:11.933ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
872026-09-04T17:59:11.936ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
882026-09-04T17:59:13.118ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
892026-09-04T17:59:13.123ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
902026-09-04T17:59:13.126ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
912026-09-04T17:59:14.414ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
922026-09-04T17:59:14.420ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
932026-09-04T17:59:14.423ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
942026-09-04T17:59:15.563ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
952026-09-04T17:59:15.567ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
962026-09-04T17:59:15.570ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
972026-09-04T17:59:15.597ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16506387734860978422 n_samples = 6 unit = oximeter
982026-09-04T17:59:16.161ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
992026-09-04T17:59:16.166ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1002026-09-04T17:59:16.169ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1012026-09-04T17:59:16.755ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1022026-09-04T17:59:16.759ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1032026-09-04T17:59:16.763ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1042026-09-04T17:59:17.464ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1052026-09-04T17:59:17.468ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1062026-09-04T17:59:17.471ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1072026-09-04T17:59:18.716ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1082026-09-04T17:59:18.720ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1092026-09-04T17:59:18.723ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1102026-09-04T17:59:19.959ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 798518086241873458 n_samples = 6 unit = oximeter
1112026-09-04T17:59:20.221ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-09-04T17:59:20.225ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1132026-09-04T17:59:20.229ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1142026-09-04T17:59:20.944ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-09-04T17:59:20.949ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1162026-09-04T17:59:20.952ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1172026-09-04T17:59:21.786ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1182026-09-04T17:59:21.795ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1192026-09-04T17:59:21.799ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1202026-09-04T17:59:23.057ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1212026-09-04T17:59:23.062ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1222026-09-04T17:59:23.066ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1232026-09-04T17:59:23.861ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1242026-09-04T17:59:23.867ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1252026-09-04T17:59:23.871ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1262026-09-04T17:59:24.978ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16506387734860978422 n_samples = 6 unit = oximeter
1272026-09-04T17:59:24.990ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1282026-09-04T17:59:24.991ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 20.023952111s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1292026-09-04T17:59:25.190ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1302026-09-04T17:59:25.196ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1312026-09-04T17:59:25.199ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1322026-09-04T17:59:25.308ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("34c992a0-d2ce-4be5-b32a-57ee6226b298.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9380dd25-afd7-42dc-b652-d3dd3b4da0a2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ba8fe925-d548-45b6-922d-69ed5d6bc63d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ba8fe925-d548-45b6-922d-69ed5d6bc63d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 773, tv_nsec: 761098870 } }) unit = producer-server
1332026-09-04T17:59:25.311ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1342026-09-04T17:59:25.320ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1352026-09-04T17:59:26.609ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1362026-09-04T17:59:26.616ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1372026-09-04T17:59:26.621ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1382026-09-04T17:59:27.824ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1392026-09-04T17:59:27.829ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1402026-09-04T17:59:27.833ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1412026-09-04T17:59:28.830ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1422026-09-04T17:59:28.835ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1432026-09-04T17:59:28.838ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1442026-09-04T17:59:29.328ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 798518086241873458 n_samples = 6 unit = oximeter
1452026-09-04T17:59:30.048ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-09-04T17:59:30.053ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1472026-09-04T17:59:30.056ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1482026-09-04T17:59:31.307ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-09-04T17:59:31.312ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1502026-09-04T17:59:31.314ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1512026-09-04T17:59:32.655ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-09-04T17:59:32.661ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1532026-09-04T17:59:32.664ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1542026-09-04T17:59:34.142ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1552026-09-04T17:59:34.147ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1562026-09-04T17:59:34.150ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1572026-09-04T17:59:34.827ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1582026-09-04T17:59:34.831ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1592026-09-04T17:59:34.834ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1602026-09-04T17:59:34.953ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16506387734860978422 n_samples = 6 unit = oximeter
1612026-09-04T17:59:36.144ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1622026-09-04T17:59:36.149ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1632026-09-04T17:59:36.152ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1642026-09-04T17:59:36.947ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1652026-09-04T17:59:36.952ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1662026-09-04T17:59:36.955ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1672026-09-04T17:59:38.422ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1682026-09-04T17:59:38.427ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1692026-09-04T17:59:38.430ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1702026-09-04T17:59:39.304ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 798518086241873458 n_samples = 6 unit = oximeter
1712026-09-04T17:59:39.422ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-09-04T17:59:39.430ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1732026-09-04T17:59:39.433ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1742026-09-04T17:59:40.076ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1752026-09-04T17:59:40.081ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1762026-09-04T17:59:40.084ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1772026-09-04T17:59:40.281ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1782026-09-04T17:59:40.282ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 35.3556661s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1792026-09-04T17:59:40.612ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("34c992a0-d2ce-4be5-b32a-57ee6226b298.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9380dd25-afd7-42dc-b652-d3dd3b4da0a2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ba8fe925-d548-45b6-922d-69ed5d6bc63d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ba8fe925-d548-45b6-922d-69ed5d6bc63d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 789, tv_nsec: 106327944 } }) unit = producer-server
1802026-09-04T17:59:40.612ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1812026-09-04T17:59:40.619ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1822026-09-04T17:59:41.000ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1832026-09-04T17:59:41.004ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1842026-09-04T17:59:41.007ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1852026-09-04T17:59:42.470ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862026-09-04T17:59:42.476ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1872026-09-04T17:59:42.480ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1882026-09-04T17:59:43.613ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-09-04T17:59:43.618ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1902026-09-04T17:59:43.622ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1912026-09-04T17:59:45.048ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16506387734860978422 n_samples = 6 unit = oximeter
1922026-09-04T17:59:45.048ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1932026-09-04T17:59:45.061ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1942026-09-04T17:59:45.074ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1952026-09-04T17:59:45.714ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1962026-09-04T17:59:45.722ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
1972026-09-04T17:59:45.726ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1982026-09-04T17:59:46.367ZINFOtfportd: removed stale NDP entry :fe80::ccd9:1fff:fe55:ad8b->ce:d9:1f:55:ad:8b
1992026-09-04T17:59:47.113ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-09-04T17:59:47.122ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2012026-09-04T17:59:47.127ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2022026-09-04T17:59:47.837ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::ccd9:1fff:fe55:ad8b->ce:d9:1f:55:ad:8b interface = tfportrear0_0
2032026-09-04T17:59:48.548ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2042026-09-04T17:59:48.553ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2052026-09-04T17:59:48.556ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2062026-09-04T17:59:49.279ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 798518086241873458 n_samples = 6 unit = oximeter
2072026-09-04T17:59:49.348ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2082026-09-04T17:59:49.353ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2092026-09-04T17:59:49.357ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2102026-09-04T17:59:50.337ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2112026-09-04T17:59:50.343ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2122026-09-04T17:59:50.346ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2132026-09-04T17:59:50.856ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2142026-09-04T17:59:50.861ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2152026-09-04T17:59:50.864ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2162026-09-04T17:59:51.945ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2172026-09-04T17:59:51.949ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2182026-09-04T17:59:51.952ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2192026-09-04T17:59:52.507ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2202026-09-04T17:59:52.513ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2212026-09-04T17:59:52.516ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2222026-09-04T17:59:53.558ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2232026-09-04T17:59:53.563ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2242026-09-04T17:59:53.566ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2252026-09-04T17:59:54.634ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2262026-09-04T17:59:54.639ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2272026-09-04T17:59:54.642ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2282026-09-04T17:59:55.005ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16506387734860978422 n_samples = 6 unit = oximeter
2292026-09-04T17:59:55.580ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2302026-09-04T17:59:55.581ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 50.695784231s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2312026-09-04T17:59:56.121ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2322026-09-04T17:59:56.127ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2332026-09-04T17:59:56.130ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2342026-09-04T17:59:56.699ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("34c992a0-d2ce-4be5-b32a-57ee6226b298.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9380dd25-afd7-42dc-b652-d3dd3b4da0a2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ba8fe925-d548-45b6-922d-69ed5d6bc63d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ba8fe925-d548-45b6-922d-69ed5d6bc63d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 805, tv_nsec: 141284482 } }) unit = producer-server
2352026-09-04T17:59:56.704ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
2362026-09-04T17:59:56.722ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
2372026-09-04T17:59:56.891ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2382026-09-04T17:59:56.902ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2392026-09-04T17:59:56.907ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2402026-09-04T17:59:58.217ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2412026-09-04T17:59:58.222ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2422026-09-04T17:59:58.225ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2432026-09-04T17:59:58.974ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2442026-09-04T17:59:58.979ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2452026-09-04T17:59:58.982ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2462026-09-04T17:59:59.253ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 798518086241873458 n_samples = 6 unit = oximeter
2472026-09-04T17:59:59.883ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2482026-09-04T17:59:59.889ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::f463:c0ff:feec:668f]
2492026-09-04T17:59:59.892ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]