Max level shown:
1 [ Jun 18 17:12:53 Enabled. ]
2 [ Jun 18 17:12:53 Rereading configuration. ]
3 [ Jun 18 17:12:54 Rereading configuration. ]
4 [ Jun 18 17:13:15 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jun 18 17:13:15 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-06-18T17:13:15.534ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-06-18T17:13:15.536ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
92026-06-18T17:13:15.537ZINFOtfportd: running in sync-only mode
102026-06-18T17:13:15.537ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-06-18T17:13:15.539ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-06-18T17:13:15.541ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 9a7f88d0-8413-4dbb-95c9-dd91f7c5f894, ), sled_id: Some( 3d4e0c89-13ee-4534-8ce4-b5b47ce2a054, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
142026-06-18T17:13:15.541ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-06-18T17:13:15.545ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::c083:e8ff:feb6:f511->c2:83:e8:b6:f5:11 interface = tfportrear0_0
162026-06-18T17:13:15.546ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-06-18T17:13:15.552ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
182026-06-18T17:13:15.558ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Jun 18 17:13:16 Rereading configuration. ]
20 [ Jun 18 17:13:16 Executing refresh method (:kill). ]
21 [ Jun 18 17:13:16 Stopping because all processes in service exited. ]
22 [ Jun 18 17:13:16 Executing stop method (:kill). ]
23 [ Jun 18 17:13:16 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Jun 18 17:13:16 Method "start" exited with status 0. ]
252026-06-18T17:13:16.209ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-06-18T17:13:16.210ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
272026-06-18T17:13:16.211ZINFOtfportd: running in sync-only mode
282026-06-18T17:13:16.211ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-06-18T17:13:16.213ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-06-18T17:13:16.213ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 9a7f88d0-8413-4dbb-95c9-dd91f7c5f894, sled_id: 3d4e0c89-13ee-4534-8ce4-b5b47ce2a054, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: a9a5994a-2d92-4a69-bdf6-7b3ffc8c99a6, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-06-18T17:13:16.213ZDEBGtfportd: created producer ID producer_id = 0e941d13-ba6f-42e6-a4db-131a77abc166 unit = oximeter
322026-06-18T17:13:16.214ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 9a7f88d0-8413-4dbb-95c9-dd91f7c5f894, ), sled_id: Some( 3d4e0c89-13ee-4534-8ce4-b5b47ce2a054, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
332026-06-18T17:13:16.221ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-06-18T17:13:16.223ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::c083:e8ff:feb6:f511->c2:83:e8:b6:f5:11 interface = tfportrear0_0
352026-06-18T17:13:16.223ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::205c:76ff:fe8b:ff3a->22:5c:76:8b:ff:3a interface = tfportrear0_0
362026-06-18T17:13:16.226ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-06-18T17:13:16.226ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:56087 method = GET path = /{producer_id} unit = producer-server versions = all
382026-06-18T17:13:16.226ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:56087 unit = producer-server
392026-06-18T17:13:16.227ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:56087 unit = producer-server
402026-06-18T17:13:16.227ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:56087 unit = producer-server
412026-06-18T17:13:16.227ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-06-18T17:13:16.227ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-06-18T17:13:16.227ZDEBGtfportd: starting producer registration task unit = producer-server
442026-06-18T17:13:16.227ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:56087 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 0e941d13-ba6f-42e6-a4db-131a77abc166 unit = producer-server
452026-06-18T17:13:16.227ZINFOtfportd: created oximeter producer server unit = oximeter
462026-06-18T17:13:16.227ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-06-18T17:13:16.227ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
482026-06-18T17:13:16.228ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-06-18T17:13:16.229ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1b4537ae-094c-497c-a84f-b5ad382eb6ad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("24141d23-436b-4751-9347-9d28621d62fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 683, tv_nsec: 880334127 } }) unit = producer-server
502026-06-18T17:13:16.229ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-06-18T17:12:49.400964292Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
512026-06-18T17:13:16.229ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 6402500496168290892 unit = oximeter
522026-06-18T17:13:16.229ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
532026-06-18T17:13:16.230ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
542026-06-18T17:13:16.230ZWARNtfportd: tfportqsfp0_0 has no addrs
552026-06-18T17:13:16.250ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
562026-06-18T17:13:16.957ZINFOtfportd: added new ARP entry entry = 10.151.2.174->9e:86:a1:91:9f:69 interface = tfportqsfp0_0
572026-06-18T17:13:17.578ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
582026-06-18T17:13:17.584ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
592026-06-18T17:13:17.587ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
602026-06-18T17:13:17.587ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
612026-06-18T17:13:17.589ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-06-18T17:12:49.400833141Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
622026-06-18T17:13:17.589ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 9145060946436028062 unit = oximeter
632026-06-18T17:13:17.589ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
642026-06-18T17:13:18.450ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-06-18T17:13:18.455ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
662026-06-18T17:13:18.458ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
672026-06-18T17:13:19.543ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-06-18T17:13:19.549ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
692026-06-18T17:13:19.552ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
702026-06-18T17:13:20.449ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-06-18T17:13:20.458ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
722026-06-18T17:13:20.460ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
732026-06-18T17:13:21.598ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-06-18T17:13:21.604ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
752026-06-18T17:13:21.607ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
762026-06-18T17:13:22.865ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-06-18T17:13:22.870ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
782026-06-18T17:13:22.873ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
792026-06-18T17:13:23.539ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-06-18T17:13:23.543ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
812026-06-18T17:13:23.547ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
822026-06-18T17:13:24.079ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-06-18T17:13:24.083ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
842026-06-18T17:13:24.086ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
852026-06-18T17:13:25.591ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862026-06-18T17:13:25.596ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
872026-06-18T17:13:25.600ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
882026-06-18T17:13:26.233ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6402500496168290892 n_samples = 6 unit = oximeter
892026-06-18T17:13:26.358ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902026-06-18T17:13:26.361ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
912026-06-18T17:13:26.364ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
922026-06-18T17:13:26.946ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
932026-06-18T17:13:26.951ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
942026-06-18T17:13:26.954ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
952026-06-18T17:13:27.592ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9145060946436028062 n_samples = 6 unit = oximeter
962026-06-18T17:13:27.861ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-06-18T17:13:27.866ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
982026-06-18T17:13:27.869ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
992026-06-18T17:13:28.971ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002026-06-18T17:13:28.976ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1012026-06-18T17:13:28.979ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1022026-06-18T17:13:29.632ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-06-18T17:13:29.637ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1042026-06-18T17:13:29.640ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1052026-06-18T17:13:29.692ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-06-18T17:13:29.696ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1072026-06-18T17:13:29.699ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1082026-06-18T17:13:30.563ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1092026-06-18T17:13:30.565ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 15.025001441s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1102026-06-18T17:13:30.713ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1112026-06-18T17:13:30.718ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1122026-06-18T17:13:30.718ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1b4537ae-094c-497c-a84f-b5ad382eb6ad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("24141d23-436b-4751-9347-9d28621d62fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 699, tv_nsec: 56953977 } }) unit = producer-server
1132026-06-18T17:13:30.719ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1142026-06-18T17:13:30.727ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1152026-06-18T17:13:30.730ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1162026-06-18T17:13:32.034ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1172026-06-18T17:13:32.039ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1182026-06-18T17:13:32.043ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1192026-06-18T17:13:33.141ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1202026-06-18T17:13:33.146ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1212026-06-18T17:13:33.150ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1222026-06-18T17:13:34.542ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1232026-06-18T17:13:34.547ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1242026-06-18T17:13:34.550ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1252026-06-18T17:13:35.561ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6402500496168290892 n_samples = 6 unit = oximeter
1262026-06-18T17:13:35.855ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1272026-06-18T17:13:35.860ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1282026-06-18T17:13:35.863ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1292026-06-18T17:13:36.417ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1302026-06-18T17:13:36.421ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1312026-06-18T17:13:36.424ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1322026-06-18T17:13:36.924ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9145060946436028062 n_samples = 6 unit = oximeter
1332026-06-18T17:13:37.031ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342026-06-18T17:13:37.036ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1352026-06-18T17:13:37.039ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1362026-06-18T17:13:38.239ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-06-18T17:13:38.243ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1382026-06-18T17:13:38.246ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1392026-06-18T17:13:38.759ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-06-18T17:13:38.763ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1412026-06-18T17:13:38.767ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1422026-06-18T17:13:39.418ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-06-18T17:13:39.422ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1442026-06-18T17:13:39.426ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1452026-06-18T17:13:40.063ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-06-18T17:13:40.067ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1472026-06-18T17:13:40.071ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1482026-06-18T17:13:41.373ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-06-18T17:13:41.377ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1502026-06-18T17:13:41.380ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1512026-06-18T17:13:42.478ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-06-18T17:13:42.483ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1532026-06-18T17:13:42.487ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1542026-06-18T17:13:43.695ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1552026-06-18T17:13:43.701ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1562026-06-18T17:13:43.704ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1572026-06-18T17:13:44.551ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1582026-06-18T17:13:44.555ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1592026-06-18T17:13:44.558ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1602026-06-18T17:13:45.590ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6402500496168290892 n_samples = 6 unit = oximeter
1612026-06-18T17:13:45.769ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1622026-06-18T17:13:45.770ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 30.188859696s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1632026-06-18T17:13:45.921ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1642026-06-18T17:13:45.926ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1652026-06-18T17:13:45.930ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1662026-06-18T17:13:46.318ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1b4537ae-094c-497c-a84f-b5ad382eb6ad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("24141d23-436b-4751-9347-9d28621d62fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 714, tv_nsec: 613916524 } }) unit = producer-server
1672026-06-18T17:13:46.319ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1682026-06-18T17:13:46.327ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1692026-06-18T17:13:46.953ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9145060946436028062 n_samples = 6 unit = oximeter
1702026-06-18T17:13:47.132ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1712026-06-18T17:13:47.137ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1722026-06-18T17:13:47.140ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1732026-06-18T17:13:47.940ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1742026-06-18T17:13:47.945ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1752026-06-18T17:13:47.948ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1762026-06-18T17:13:48.569ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1772026-06-18T17:13:48.574ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1782026-06-18T17:13:48.577ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1792026-06-18T17:13:49.570ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1802026-06-18T17:13:49.575ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1812026-06-18T17:13:49.578ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1822026-06-18T17:13:50.747ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1832026-06-18T17:13:50.752ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1842026-06-18T17:13:50.755ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1852026-06-18T17:13:51.363ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862026-06-18T17:13:51.367ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1872026-06-18T17:13:51.370ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1882026-06-18T17:13:52.410ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-06-18T17:13:52.415ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1902026-06-18T17:13:52.418ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1912026-06-18T17:13:53.850ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1922026-06-18T17:13:53.855ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1932026-06-18T17:13:53.859ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1942026-06-18T17:13:54.902ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1952026-06-18T17:13:54.906ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
1962026-06-18T17:13:54.910ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1972026-06-18T17:13:55.619ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6402500496168290892 n_samples = 6 unit = oximeter
1982026-06-18T17:13:56.308ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1992026-06-18T17:13:56.311ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2002026-06-18T17:13:56.315ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2012026-06-18T17:13:56.982ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9145060946436028062 n_samples = 6 unit = oximeter
2022026-06-18T17:13:57.749ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032026-06-18T17:13:57.753ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2042026-06-18T17:13:57.756ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2052026-06-18T17:13:58.555ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2062026-06-18T17:13:58.559ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2072026-06-18T17:13:58.562ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2082026-06-18T17:13:59.640ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-06-18T17:13:59.645ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2102026-06-18T17:13:59.647ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2112026-06-18T17:14:00.196ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-06-18T17:14:00.200ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2132026-06-18T17:14:00.204ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2142026-06-18T17:14:01.262ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2152026-06-18T17:14:01.267ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2162026-06-18T17:14:01.270ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2172026-06-18T17:14:01.369ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2182026-06-18T17:14:01.371ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 45.747463519s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2192026-06-18T17:14:01.946ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1b4537ae-094c-497c-a84f-b5ad382eb6ad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("24141d23-436b-4751-9347-9d28621d62fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 730, tv_nsec: 200057270 } }) unit = producer-server
2202026-06-18T17:14:01.947ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
2212026-06-18T17:14:01.955ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
2222026-06-18T17:14:02.023ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2232026-06-18T17:14:02.027ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2242026-06-18T17:14:02.030ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2252026-06-18T17:14:03.077ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2262026-06-18T17:14:03.081ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2272026-06-18T17:14:03.084ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2282026-06-18T17:14:04.262ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2292026-06-18T17:14:04.267ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2302026-06-18T17:14:04.270ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2312026-06-18T17:14:05.573ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2322026-06-18T17:14:05.578ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2332026-06-18T17:14:05.581ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2342026-06-18T17:14:05.648ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6402500496168290892 n_samples = 6 unit = oximeter
2352026-06-18T17:14:06.169ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2362026-06-18T17:14:06.174ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2372026-06-18T17:14:06.178ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2382026-06-18T17:14:07.011ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9145060946436028062 n_samples = 6 unit = oximeter
2392026-06-18T17:14:07.518ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402026-06-18T17:14:07.524ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2412026-06-18T17:14:07.527ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2422026-06-18T17:14:08.930ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2432026-06-18T17:14:08.935ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2442026-06-18T17:14:08.938ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2452026-06-18T17:14:09.758ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462026-06-18T17:14:09.762ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2472026-06-18T17:14:09.766ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2482026-06-18T17:14:11.264ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2492026-06-18T17:14:11.269ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2502026-06-18T17:14:11.272ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2512026-06-18T17:14:12.744ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2522026-06-18T17:14:12.748ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2532026-06-18T17:14:12.752ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2542026-06-18T17:14:13.277ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2552026-06-18T17:14:13.282ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2562026-06-18T17:14:13.285ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2572026-06-18T17:14:14.657ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2582026-06-18T17:14:14.661ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2592026-06-18T17:14:14.664ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2602026-06-18T17:14:15.446ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2612026-06-18T17:14:15.451ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2622026-06-18T17:14:15.454ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2632026-06-18T17:14:15.677ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6402500496168290892 n_samples = 6 unit = oximeter
2642026-06-18T17:14:16.637ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2652026-06-18T17:14:16.642ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2662026-06-18T17:14:16.645ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2672026-06-18T17:14:16.996ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2682026-06-18T17:14:16.997ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 61.331283348s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2692026-06-18T17:14:17.040ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9145060946436028062 n_samples = 6 unit = oximeter
2702026-06-18T17:14:17.609ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2712026-06-18T17:14:17.614ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2722026-06-18T17:14:17.617ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2732026-06-18T17:14:18.215ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2742026-06-18T17:14:18.220ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2752026-06-18T17:14:18.223ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2762026-06-18T17:14:19.284ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2772026-06-18T17:14:19.289ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2782026-06-18T17:14:19.292ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2792026-06-18T17:14:19.455ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1b4537ae-094c-497c-a84f-b5ad382eb6ad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("24141d23-436b-4751-9347-9d28621d62fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 747, tv_nsec: 661600513 } }) unit = producer-server
2802026-06-18T17:14:19.456ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
2812026-06-18T17:14:19.464ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
2822026-06-18T17:14:19.903ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2832026-06-18T17:14:19.909ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2842026-06-18T17:14:19.911ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2852026-06-18T17:14:20.550ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2862026-06-18T17:14:20.555ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2872026-06-18T17:14:20.558ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2882026-06-18T17:14:21.916ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2892026-06-18T17:14:21.922ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2902026-06-18T17:14:21.924ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2912026-06-18T17:14:23.411ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2922026-06-18T17:14:23.416ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2932026-06-18T17:14:23.419ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2942026-06-18T17:14:24.348ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2952026-06-18T17:14:24.353ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2962026-06-18T17:14:24.356ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2972026-06-18T17:14:24.962ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2982026-06-18T17:14:24.967ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
2992026-06-18T17:14:24.970ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3002026-06-18T17:14:25.706ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6402500496168290892 n_samples = 6 unit = oximeter
3012026-06-18T17:14:25.907ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3022026-06-18T17:14:25.912ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3032026-06-18T17:14:25.915ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3042026-06-18T17:14:27.000ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3052026-06-18T17:14:27.005ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3062026-06-18T17:14:27.008ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3072026-06-18T17:14:27.069ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9145060946436028062 n_samples = 6 unit = oximeter
3082026-06-18T17:14:27.869ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3092026-06-18T17:14:27.873ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3102026-06-18T17:14:27.876ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3112026-06-18T17:14:28.548ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3122026-06-18T17:14:28.553ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3132026-06-18T17:14:28.556ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3142026-06-18T17:14:29.663ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3152026-06-18T17:14:29.668ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3162026-06-18T17:14:29.671ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3172026-06-18T17:14:30.228ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3182026-06-18T17:14:30.234ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3192026-06-18T17:14:30.237ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3202026-06-18T17:14:30.795ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3212026-06-18T17:14:30.799ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3222026-06-18T17:14:30.802ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3232026-06-18T17:14:31.335ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3242026-06-18T17:14:31.340ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3252026-06-18T17:14:31.344ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3262026-06-18T17:14:32.202ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3272026-06-18T17:14:32.207ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3282026-06-18T17:14:32.210ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3292026-06-18T17:14:33.076ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3302026-06-18T17:14:33.080ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3312026-06-18T17:14:33.083ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3322026-06-18T17:14:34.020ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3332026-06-18T17:14:34.024ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3342026-06-18T17:14:34.027ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3352026-06-18T17:14:34.505ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3362026-06-18T17:14:34.506ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 78.793631335s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3372026-06-18T17:14:35.294ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382026-06-18T17:14:35.299ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3392026-06-18T17:14:35.302ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3402026-06-18T17:14:35.736ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6402500496168290892 n_samples = 6 unit = oximeter
3412026-06-18T17:14:36.090ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3422026-06-18T17:14:36.094ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3432026-06-18T17:14:36.097ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3442026-06-18T17:14:36.682ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3452026-06-18T17:14:36.687ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3462026-06-18T17:14:36.690ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3472026-06-18T17:14:37.098ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9145060946436028062 n_samples = 6 unit = oximeter
3482026-06-18T17:14:37.850ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3492026-06-18T17:14:37.854ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3502026-06-18T17:14:37.858ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3512026-06-18T17:14:38.035ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1b4537ae-094c-497c-a84f-b5ad382eb6ad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("24141d23-436b-4751-9347-9d28621d62fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("57775cc3-660c-4770-ad47-5c8b87561c9e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 766, tv_nsec: 191190588 } }) unit = producer-server
3522026-06-18T17:14:38.036ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
3532026-06-18T17:14:38.043ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
3542026-06-18T17:14:38.409ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3552026-06-18T17:14:38.414ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3562026-06-18T17:14:38.417ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3572026-06-18T17:14:39.599ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3582026-06-18T17:14:39.604ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3592026-06-18T17:14:39.607ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3602026-06-18T17:14:40.672ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3612026-06-18T17:14:40.677ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3622026-06-18T17:14:40.680ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3632026-06-18T17:14:41.502ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3642026-06-18T17:14:41.506ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3652026-06-18T17:14:41.509ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3662026-06-18T17:14:42.936ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3672026-06-18T17:14:42.941ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3682026-06-18T17:14:42.945ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3692026-06-18T17:14:43.972ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3702026-06-18T17:14:43.978ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3712026-06-18T17:14:43.981ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3722026-06-18T17:14:45.342ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3732026-06-18T17:14:45.347ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3742026-06-18T17:14:45.350ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3752026-06-18T17:14:45.765ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6402500496168290892 n_samples = 6 unit = oximeter
3762026-06-18T17:14:45.966ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3772026-06-18T17:14:45.970ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3782026-06-18T17:14:45.973ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3792026-06-18T17:14:46.933ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3802026-06-18T17:14:46.938ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3812026-06-18T17:14:46.941ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3822026-06-18T17:14:47.127ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9145060946436028062 n_samples = 6 unit = oximeter
3832026-06-18T17:14:47.901ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3842026-06-18T17:14:47.906ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3852026-06-18T17:14:47.909ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3862026-06-18T17:14:48.779ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3872026-06-18T17:14:48.784ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3882026-06-18T17:14:48.787ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3892026-06-18T17:14:50.284ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3902026-06-18T17:14:50.288ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3912026-06-18T17:14:50.292ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3922026-06-18T17:14:51.324ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3932026-06-18T17:14:51.328ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3942026-06-18T17:14:51.331ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3952026-06-18T17:14:52.697ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3962026-06-18T17:14:52.702ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
3972026-06-18T17:14:52.705ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3982026-06-18T17:14:53.084ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3992026-06-18T17:14:53.086ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 97.322739281s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
4002026-06-18T17:14:53.298ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4012026-06-18T17:14:53.302ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
4022026-06-18T17:14:53.306ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4032026-06-18T17:14:54.036ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4042026-06-18T17:14:54.040ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
4052026-06-18T17:14:54.043ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4062026-06-18T17:14:54.570ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4072026-06-18T17:14:54.575ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
4082026-06-18T17:14:54.578ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4092026-06-18T17:14:55.722ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4102026-06-18T17:14:55.727ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
4112026-06-18T17:14:55.730ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4122026-06-18T17:14:55.794ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6402500496168290892 n_samples = 6 unit = oximeter
4132026-06-18T17:14:56.795ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4142026-06-18T17:14:56.799ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
4152026-06-18T17:14:56.802ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4162026-06-18T17:14:57.157ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9145060946436028062 n_samples = 6 unit = oximeter
4172026-06-18T17:14:57.318ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4182026-06-18T17:14:57.323ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
4192026-06-18T17:14:57.326ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4202026-06-18T17:14:57.968ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4212026-06-18T17:14:57.973ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
4222026-06-18T17:14:57.975ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4232026-06-18T17:14:59.151ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4242026-06-18T17:14:59.155ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
4252026-06-18T17:14:59.158ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4262026-06-18T17:15:00.266ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4272026-06-18T17:15:00.271ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::c083:e8ff:feb6:f511]
4282026-06-18T17:15:00.275ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]