Max level shown:
1 [ Jun 11 21:27:47 Enabled. ]
2 [ Jun 11 21:27:47 Rereading configuration. ]
3 [ Jun 11 21:27:48 Rereading configuration. ]
4 [ Jun 11 21:28:09 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jun 11 21:28:09 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-06-11T21:28:09.339ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-06-11T21:28:09.341ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
92026-06-11T21:28:09.341ZINFOtfportd: running in sync-only mode
102026-06-11T21:28:09.341ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-06-11T21:28:09.343ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-06-11T21:28:09.345ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 10ee1c88-be57-4244-a2e9-5da2a31f4b88, ), sled_id: Some( b3ce080e-3857-44e8-a6ae-32572e704bfa, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-06-11T21:28:09.345ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-06-11T21:28:09.349ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::142f:5cff:fe72:49d1->16:2f:5c:72:49:d1 interface = tfportrear0_0
162026-06-11T21:28:09.351ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-06-11T21:28:09.357ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
182026-06-11T21:28:09.362ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Jun 11 21:28:09 Rereading configuration. ]
20 [ Jun 11 21:28:09 Executing refresh method (:kill). ]
21 [ Jun 11 21:28:09 Stopping because all processes in service exited. ]
22 [ Jun 11 21:28:09 Executing stop method (:kill). ]
23 [ Jun 11 21:28:09 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Jun 11 21:28:09 Method "start" exited with status 0. ]
252026-06-11T21:28:09.988ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-06-11T21:28:09.989ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
272026-06-11T21:28:09.990ZINFOtfportd: running in sync-only mode
282026-06-11T21:28:09.990ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-06-11T21:28:09.992ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-06-11T21:28:09.992ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 10ee1c88-be57-4244-a2e9-5da2a31f4b88, sled_id: b3ce080e-3857-44e8-a6ae-32572e704bfa, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 309b8e9e-f43e-44a4-977f-d8da9ca30a87, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-06-11T21:28:09.992ZDEBGtfportd: created producer ID producer_id = 93bb9a18-723e-4208-9338-b72f11cc0af5 unit = oximeter
322026-06-11T21:28:09.993ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 10ee1c88-be57-4244-a2e9-5da2a31f4b88, ), sled_id: Some( b3ce080e-3857-44e8-a6ae-32572e704bfa, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
332026-06-11T21:28:10.000ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-06-11T21:28:10.002ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::142f:5cff:fe72:49d1->16:2f:5c:72:49:d1 interface = tfportrear0_0
352026-06-11T21:28:10.002ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::8042:26ff:fe11:649b->82:42:26:11:64:9b interface = tfportrear0_0
362026-06-11T21:28:10.003ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-06-11T21:28:10.004ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:64944 method = GET path = /{producer_id} unit = producer-server versions = all
382026-06-11T21:28:10.005ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:64944 unit = producer-server
392026-06-11T21:28:10.005ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:64944 unit = producer-server
402026-06-11T21:28:10.005ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:64944 unit = producer-server
412026-06-11T21:28:10.005ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-06-11T21:28:10.005ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-06-11T21:28:10.005ZDEBGtfportd: starting producer registration task unit = producer-server
442026-06-11T21:28:10.005ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:64944 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 93bb9a18-723e-4208-9338-b72f11cc0af5 unit = producer-server
452026-06-11T21:28:10.005ZINFOtfportd: created oximeter producer server unit = oximeter
462026-06-11T21:28:10.005ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-06-11T21:28:10.005ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
482026-06-11T21:28:10.006ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-06-11T21:28:10.008ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-06-11T21:27:43.128622078Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502026-06-11T21:28:10.008ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 10046019443634843584 unit = oximeter
512026-06-11T21:28:10.008ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522026-06-11T21:28:10.009ZWARNtfportd: tfportqsfp0_0 has no addrs
532026-06-11T21:28:10.928ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
542026-06-11T21:28:10.933ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
552026-06-11T21:28:10.936ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
562026-06-11T21:28:10.937ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
572026-06-11T21:28:10.937ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-06-11T21:27:43.128492077Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
582026-06-11T21:28:10.937ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 6575611609766352868 unit = oximeter
592026-06-11T21:28:10.937ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
602026-06-11T21:28:11.108ZINFOtfportd: added new ARP entry entry = 10.151.1.174->8a:5e:5d:4a:87:20 interface = tfportqsfp0_0
612026-06-11T21:28:11.720ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-06-11T21:28:11.724ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
632026-06-11T21:28:11.727ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
642026-06-11T21:28:13.132ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-06-11T21:28:13.138ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
662026-06-11T21:28:13.141ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
672026-06-11T21:28:14.284ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-06-11T21:28:14.289ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
692026-06-11T21:28:14.292ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
702026-06-11T21:28:15.757ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-06-11T21:28:15.762ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
722026-06-11T21:28:15.765ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
732026-06-11T21:28:16.716ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-06-11T21:28:16.720ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
752026-06-11T21:28:16.723ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
762026-06-11T21:28:17.858ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-06-11T21:28:17.863ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
782026-06-11T21:28:17.866ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
792026-06-11T21:28:18.487ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-06-11T21:28:18.492ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
812026-06-11T21:28:18.495ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
822026-06-11T21:28:19.813ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-06-11T21:28:19.819ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
842026-06-11T21:28:19.822ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
852026-06-11T21:28:20.010ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter
862026-06-11T21:28:20.011ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("51f12723-3ce8-402d-b7c8-b003c448798c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("52be77ee-b420-4ff7-9e35-c2de10747fba.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 743, tv_nsec: 481303675 } }) unit = producer-server
872026-06-11T21:28:20.012ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
882026-06-11T21:28:20.028ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
892026-06-11T21:28:20.940ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6575611609766352868 n_samples = 6 unit = oximeter
902026-06-11T21:28:21.062ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
912026-06-11T21:28:21.067ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
922026-06-11T21:28:21.070ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
932026-06-11T21:28:22.027ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942026-06-11T21:28:22.031ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
952026-06-11T21:28:22.034ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
962026-06-11T21:28:22.608ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-06-11T21:28:22.612ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
982026-06-11T21:28:22.615ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
992026-06-11T21:28:23.576ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002026-06-11T21:28:23.582ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1012026-06-11T21:28:23.585ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1022026-06-11T21:28:24.223ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-06-11T21:28:24.229ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1042026-06-11T21:28:24.232ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1052026-06-11T21:28:25.103ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-06-11T21:28:25.108ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1072026-06-11T21:28:25.111ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1082026-06-11T21:28:26.011ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-06-11T21:28:26.014ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1102026-06-11T21:28:26.018ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1112026-06-11T21:28:26.870ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-06-11T21:28:26.874ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1132026-06-11T21:28:26.878ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1142026-06-11T21:28:28.346ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-06-11T21:28:28.351ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1162026-06-11T21:28:28.354ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1172026-06-11T21:28:29.601ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1182026-06-11T21:28:29.605ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1192026-06-11T21:28:29.608ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1202026-06-11T21:28:30.013ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter
1212026-06-11T21:28:30.345ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1222026-06-11T21:28:30.351ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1232026-06-11T21:28:30.354ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1242026-06-11T21:28:30.942ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6575611609766352868 n_samples = 6 unit = oximeter
1252026-06-11T21:28:31.579ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1262026-06-11T21:28:31.585ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1272026-06-11T21:28:31.588ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1282026-06-11T21:28:32.407ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1292026-06-11T21:28:32.413ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1302026-06-11T21:28:32.416ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1312026-06-11T21:28:33.190ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1322026-06-11T21:28:33.195ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1332026-06-11T21:28:33.198ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1342026-06-11T21:28:33.998ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1352026-06-11T21:28:34.006ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1362026-06-11T21:28:34.009ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1372026-06-11T21:28:34.813ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1382026-06-11T21:28:34.814ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 25.02516313s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1392026-06-11T21:28:35.081ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-06-11T21:28:35.086ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1412026-06-11T21:28:35.090ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1422026-06-11T21:28:35.173ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("51f12723-3ce8-402d-b7c8-b003c448798c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("52be77ee-b420-4ff7-9e35-c2de10747fba.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 758, tv_nsec: 861749406 } }) unit = producer-server
1432026-06-11T21:28:35.174ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1442026-06-11T21:28:35.181ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1452026-06-11T21:28:35.831ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-06-11T21:28:35.836ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1472026-06-11T21:28:35.839ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1482026-06-11T21:28:36.988ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-06-11T21:28:36.992ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1502026-06-11T21:28:36.995ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1512026-06-11T21:28:38.001ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-06-11T21:28:38.006ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1532026-06-11T21:28:38.009ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1542026-06-11T21:28:38.988ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1552026-06-11T21:28:38.992ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1562026-06-11T21:28:38.996ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1572026-06-11T21:28:39.620ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1582026-06-11T21:28:39.625ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1592026-06-11T21:28:39.628ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1602026-06-11T21:28:39.784ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter
1612026-06-11T21:28:40.710ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6575611609766352868 n_samples = 6 unit = oximeter
1622026-06-11T21:28:41.055ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1632026-06-11T21:28:41.060ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1642026-06-11T21:28:41.063ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1652026-06-11T21:28:42.229ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1662026-06-11T21:28:42.233ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1672026-06-11T21:28:42.236ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1682026-06-11T21:28:42.833ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1692026-06-11T21:28:42.838ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1702026-06-11T21:28:42.842ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1712026-06-11T21:28:43.554ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-06-11T21:28:43.559ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1732026-06-11T21:28:43.562ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1742026-06-11T21:28:44.141ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1752026-06-11T21:28:44.145ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1762026-06-11T21:28:44.149ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1772026-06-11T21:28:45.445ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1782026-06-11T21:28:45.450ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1792026-06-11T21:28:45.453ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1802026-06-11T21:28:46.712ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1812026-06-11T21:28:46.717ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1822026-06-11T21:28:46.719ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1832026-06-11T21:28:47.772ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1842026-06-11T21:28:47.777ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1852026-06-11T21:28:47.780ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1862026-06-11T21:28:48.539ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1872026-06-11T21:28:48.545ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1882026-06-11T21:28:48.548ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1892026-06-11T21:28:49.657ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1902026-06-11T21:28:49.666ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
1912026-06-11T21:28:49.669ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1922026-06-11T21:28:49.759ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter
1932026-06-11T21:28:50.140ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1942026-06-11T21:28:50.142ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 40.394948171s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1952026-06-11T21:28:50.481ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("51f12723-3ce8-402d-b7c8-b003c448798c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("52be77ee-b420-4ff7-9e35-c2de10747fba.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 774, tv_nsec: 210790583 } }) unit = producer-server
1962026-06-11T21:28:50.482ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1972026-06-11T21:28:50.488ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1982026-06-11T21:28:50.686ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6575611609766352868 n_samples = 6 unit = oximeter
1992026-06-11T21:28:50.910ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-06-11T21:28:50.916ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2012026-06-11T21:28:50.919ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2022026-06-11T21:28:51.636ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032026-06-11T21:28:51.640ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2042026-06-11T21:28:51.643ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2052026-06-11T21:28:52.343ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2062026-06-11T21:28:52.347ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2072026-06-11T21:28:52.351ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2082026-06-11T21:28:53.073ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-06-11T21:28:53.078ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2102026-06-11T21:28:53.081ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2112026-06-11T21:28:54.414ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-06-11T21:28:54.418ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2132026-06-11T21:28:54.422ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2142026-06-11T21:28:55.919ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2152026-06-11T21:28:55.924ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2162026-06-11T21:28:55.927ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2172026-06-11T21:28:57.050ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2182026-06-11T21:28:57.055ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2192026-06-11T21:28:57.058ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2202026-06-11T21:28:58.074ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2212026-06-11T21:28:58.079ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2222026-06-11T21:28:58.082ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2232026-06-11T21:28:59.325ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2242026-06-11T21:28:59.330ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2252026-06-11T21:28:59.333ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2262026-06-11T21:28:59.735ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter
2272026-06-11T21:29:00.384ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2282026-06-11T21:29:00.388ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2292026-06-11T21:29:00.391ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2302026-06-11T21:29:00.662ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6575611609766352868 n_samples = 6 unit = oximeter
2312026-06-11T21:29:01.218ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2322026-06-11T21:29:01.223ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2332026-06-11T21:29:01.226ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2342026-06-11T21:29:02.561ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2352026-06-11T21:29:02.565ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2362026-06-11T21:29:02.569ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2372026-06-11T21:29:03.700ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2382026-06-11T21:29:03.706ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2392026-06-11T21:29:03.709ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2402026-06-11T21:29:04.868ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2412026-06-11T21:29:04.871ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2422026-06-11T21:29:04.874ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2432026-06-11T21:29:05.448ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2442026-06-11T21:29:05.450ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 55.744314781s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2452026-06-11T21:29:06.117ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462026-06-11T21:29:06.122ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2472026-06-11T21:29:06.125ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2482026-06-11T21:29:06.394ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("51f12723-3ce8-402d-b7c8-b003c448798c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("52be77ee-b420-4ff7-9e35-c2de10747fba.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 790, tv_nsec: 167045250 } }) unit = producer-server
2492026-06-11T21:29:06.395ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
2502026-06-11T21:29:06.403ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
2512026-06-11T21:29:06.672ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2522026-06-11T21:29:06.676ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2532026-06-11T21:29:06.680ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2542026-06-11T21:29:07.367ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2552026-06-11T21:29:07.372ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2562026-06-11T21:29:07.376ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2572026-06-11T21:29:08.100ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2582026-06-11T21:29:08.106ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2592026-06-11T21:29:08.109ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2602026-06-11T21:29:08.674ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2612026-06-11T21:29:08.680ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2622026-06-11T21:29:08.683ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2632026-06-11T21:29:09.709ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter
2642026-06-11T21:29:10.113ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2652026-06-11T21:29:10.119ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2662026-06-11T21:29:10.121ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2672026-06-11T21:29:10.637ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6575611609766352868 n_samples = 6 unit = oximeter
2682026-06-11T21:29:11.251ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692026-06-11T21:29:11.256ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2702026-06-11T21:29:11.259ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2712026-06-11T21:29:12.228ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722026-06-11T21:29:12.232ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2732026-06-11T21:29:12.235ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2742026-06-11T21:29:13.522ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2752026-06-11T21:29:13.528ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2762026-06-11T21:29:13.530ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2772026-06-11T21:29:14.686ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2782026-06-11T21:29:14.691ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2792026-06-11T21:29:14.694ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2802026-06-11T21:29:15.599ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2812026-06-11T21:29:15.602ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2822026-06-11T21:29:15.605ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2832026-06-11T21:29:16.639ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2842026-06-11T21:29:16.643ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2852026-06-11T21:29:16.647ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2862026-06-11T21:29:17.854ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2872026-06-11T21:29:17.859ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2882026-06-11T21:29:17.862ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2892026-06-11T21:29:18.541ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2902026-06-11T21:29:18.545ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2912026-06-11T21:29:18.548ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2922026-06-11T21:29:19.603ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2932026-06-11T21:29:19.607ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2942026-06-11T21:29:19.611ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2952026-06-11T21:29:19.685ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter
2962026-06-11T21:29:20.388ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2972026-06-11T21:29:20.393ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
2982026-06-11T21:29:20.397ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2992026-06-11T21:29:20.611ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6575611609766352868 n_samples = 6 unit = oximeter
3002026-06-11T21:29:21.363ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3012026-06-11T21:29:21.365ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 71.702356309s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3022026-06-11T21:29:21.658ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3032026-06-11T21:29:21.663ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3042026-06-11T21:29:21.666ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3052026-06-11T21:29:22.912ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3062026-06-11T21:29:22.917ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3072026-06-11T21:29:22.920ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3082026-06-11T21:29:23.813ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("51f12723-3ce8-402d-b7c8-b003c448798c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("52be77ee-b420-4ff7-9e35-c2de10747fba.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 807, tv_nsec: 633135301 } }) unit = producer-server
3092026-06-11T21:29:23.814ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
3102026-06-11T21:29:23.821ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
3112026-06-11T21:29:23.891ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3122026-06-11T21:29:23.896ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3132026-06-11T21:29:23.900ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3142026-06-11T21:29:24.851ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3152026-06-11T21:29:24.855ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3162026-06-11T21:29:24.859ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3172026-06-11T21:29:25.611ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3182026-06-11T21:29:25.615ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3192026-06-11T21:29:25.619ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3202026-06-11T21:29:26.209ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3212026-06-11T21:29:26.214ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3222026-06-11T21:29:26.217ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3232026-06-11T21:29:27.415ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3242026-06-11T21:29:27.419ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3252026-06-11T21:29:27.422ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3262026-06-11T21:29:28.765ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3272026-06-11T21:29:28.769ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3282026-06-11T21:29:28.772ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3292026-06-11T21:29:29.659ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter
3302026-06-11T21:29:30.005ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3312026-06-11T21:29:30.010ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3322026-06-11T21:29:30.013ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3332026-06-11T21:29:30.586ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6575611609766352868 n_samples = 6 unit = oximeter
3342026-06-11T21:29:31.219ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3352026-06-11T21:29:31.224ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3362026-06-11T21:29:31.227ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3372026-06-11T21:29:32.065ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382026-06-11T21:29:32.069ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3392026-06-11T21:29:32.072ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3402026-06-11T21:29:33.066ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3412026-06-11T21:29:33.071ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3422026-06-11T21:29:33.074ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3432026-06-11T21:29:33.712ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3442026-06-11T21:29:33.717ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3452026-06-11T21:29:33.721ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3462026-06-11T21:29:35.084ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3472026-06-11T21:29:35.089ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3482026-06-11T21:29:35.092ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3492026-06-11T21:29:36.414ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3502026-06-11T21:29:36.419ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3512026-06-11T21:29:36.422ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3522026-06-11T21:29:37.292ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3532026-06-11T21:29:37.297ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3542026-06-11T21:29:37.300ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3552026-06-11T21:29:38.704ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3562026-06-11T21:29:38.709ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3572026-06-11T21:29:38.712ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3582026-06-11T21:29:38.782ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3592026-06-11T21:29:38.783ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 89.167361369s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3602026-06-11T21:29:39.634ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter
3612026-06-11T21:29:39.728ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3622026-06-11T21:29:39.732ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3632026-06-11T21:29:39.736ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3642026-06-11T21:29:40.464ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3652026-06-11T21:29:40.468ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3662026-06-11T21:29:40.471ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3672026-06-11T21:29:40.561ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6575611609766352868 n_samples = 6 unit = oximeter
3682026-06-11T21:29:41.262ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3692026-06-11T21:29:41.267ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3702026-06-11T21:29:41.270ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3712026-06-11T21:29:42.768ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3722026-06-11T21:29:42.773ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3732026-06-11T21:29:42.776ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3742026-06-11T21:29:43.743ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3752026-06-11T21:29:43.746ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3762026-06-11T21:29:43.749ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3772026-06-11T21:29:44.024ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("51f12723-3ce8-402d-b7c8-b003c448798c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("52be77ee-b420-4ff7-9e35-c2de10747fba.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d6af5062-d40d-4f94-a3c4-61f3cbd7fb95.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 827, tv_nsec: 898522152 } }) unit = producer-server
3782026-06-11T21:29:44.025ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
3792026-06-11T21:29:44.032ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
3802026-06-11T21:29:44.673ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3812026-06-11T21:29:44.677ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3822026-06-11T21:29:44.680ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3832026-06-11T21:29:46.071ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3842026-06-11T21:29:46.076ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3852026-06-11T21:29:46.079ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3862026-06-11T21:29:46.593ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3872026-06-11T21:29:46.598ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3882026-06-11T21:29:46.601ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3892026-06-11T21:29:47.886ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3902026-06-11T21:29:47.890ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3912026-06-11T21:29:47.893ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3922026-06-11T21:29:48.625ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3932026-06-11T21:29:48.630ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3942026-06-11T21:29:48.632ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3952026-06-11T21:29:49.609ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter
3962026-06-11T21:29:50.089ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3972026-06-11T21:29:50.094ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
3982026-06-11T21:29:50.097ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3992026-06-11T21:29:50.536ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6575611609766352868 n_samples = 6 unit = oximeter
4002026-06-11T21:29:51.365ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4012026-06-11T21:29:51.371ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
4022026-06-11T21:29:51.374ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4032026-06-11T21:29:52.291ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4042026-06-11T21:29:52.295ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
4052026-06-11T21:29:52.299ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4062026-06-11T21:29:52.862ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4072026-06-11T21:29:52.868ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
4082026-06-11T21:29:52.871ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4092026-06-11T21:29:53.490ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4102026-06-11T21:29:53.495ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
4112026-06-11T21:29:53.499ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4122026-06-11T21:29:54.220ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4132026-06-11T21:29:54.225ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
4142026-06-11T21:29:54.228ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4152026-06-11T21:29:55.346ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4162026-06-11T21:29:55.351ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
4172026-06-11T21:29:55.354ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4182026-06-11T21:29:56.278ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4192026-06-11T21:29:56.282ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
4202026-06-11T21:29:56.286ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4212026-06-11T21:29:57.066ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4222026-06-11T21:29:57.071ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
4232026-06-11T21:29:57.074ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4242026-06-11T21:29:57.811ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4252026-06-11T21:29:57.816ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
4262026-06-11T21:29:57.818ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4272026-06-11T21:29:58.993ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4282026-06-11T21:29:58.994ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 109.43365177s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
4292026-06-11T21:29:59.271ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4302026-06-11T21:29:59.276ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::142f:5cff:fe72:49d1]
4312026-06-11T21:29:59.279ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4322026-06-11T21:29:59.583ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10046019443634843584 n_samples = 6 unit = oximeter