Max level shown:
1 [ Aug 31 16:27:44 Enabled. ]
2 [ Aug 31 16:27:44 Rereading configuration. ]
3 [ Aug 31 16:27:44 Rereading configuration. ]
4 [ Aug 31 16:27:56 Rereading configuration. ]
5 [ Aug 31 16:28:05 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
6 [ Aug 31 16:28:05 Method "start" exited with status 0. ]
72026-08-31T16:28:05.283ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-08-31T16:28:05.285ZERROtfportd: Failed to connect to dpd. Retrying...
92026-08-31T16:28:06.287ZINFOtfportd: connected to dpd running softnpu-3d49b131
102026-08-31T16:28:06.287ZINFOtfportd: running in sync-only mode
112026-08-31T16:28:06.289ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
122026-08-31T16:28:06.290ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
132026-08-31T16:28:06.290ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 5195c9d6-a9de-48b9-b658-111bdfc4d8e0, sled_id: b8b4d62c-e046-4d1a-ad62-6f4e824ae1cb, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 206707e3-4cc6-47a0-85a8-2b8c8e20f272, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
142026-08-31T16:28:06.290ZDEBGtfportd: created producer ID producer_id = c9461819-055e-4203-9e92-55d9d3aecb59 unit = oximeter
152026-08-31T16:28:06.291ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 5195c9d6-a9de-48b9-b658-111bdfc4d8e0, ), sled_id: Some( b8b4d62c-e046-4d1a-ad62-6f4e824ae1cb, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
162026-08-31T16:28:06.298ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-08-31T16:28:06.300ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::141c:4bff:fe9e:68d8->00:00:00:00:00:00 interface = tfportrear0_0
182026-08-31T16:28:06.301ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::d096:edff:fe57:65ab->d2:96:ed:57:65:ab interface = tfportrear0_0
192026-08-31T16:28:06.302ZDEBGtfportd: registered DTrace probes unit = producer-server
202026-08-31T16:28:06.302ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:34168 method = GET path = /{producer_id} unit = producer-server versions = all
212026-08-31T16:28:06.303ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:34168 unit = producer-server
222026-08-31T16:28:06.303ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:34168 unit = producer-server
232026-08-31T16:28:06.303ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:34168 unit = producer-server
242026-08-31T16:28:06.303ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
252026-08-31T16:28:06.303ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
262026-08-31T16:28:06.303ZDEBGtfportd: starting producer registration task unit = producer-server
272026-08-31T16:28:06.303ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:34168 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = c9461819-055e-4203-9e92-55d9d3aecb59 unit = producer-server
282026-08-31T16:28:06.303ZINFOtfportd: created oximeter producer server unit = oximeter
292026-08-31T16:28:06.303ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
302026-08-31T16:28:06.304ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
312026-08-31T16:28:06.304ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
322026-08-31T16:28:06.305ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-08-31T16:27:39.570389415Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
332026-08-31T16:28:06.305ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 4613352465068422250 unit = oximeter
342026-08-31T16:28:06.305ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2de03509-a130-46f9-b202-200f0008608c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("97eb4bac-a1da-4d58-bc01-6d83ce489de8.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 791, tv_nsec: 401058672 } }) unit = producer-server
352026-08-31T16:28:06.305ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
362026-08-31T16:28:06.306ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
372026-08-31T16:28:06.309ZWARNtfportd: tfportqsfp0_0 has no addrs
382026-08-31T16:28:06.321ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
392026-08-31T16:28:06.998ZINFOtfportd: removed stale NDP entry :fe80::141c:4bff:fe9e:68d8->00:00:00:00:00:00
402026-08-31T16:28:06.998ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::141c:4bff:fe9e:68d8->16:1c:4b:9e:68:d8 interface = tfportrear0_0
412026-08-31T16:28:07.451ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
422026-08-31T16:28:07.456ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
432026-08-31T16:28:07.460ZWARNtfportd: tfportqsfp0_0 has no addrs
442026-08-31T16:28:08.123ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
452026-08-31T16:28:08.128ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
462026-08-31T16:28:08.131ZWARNtfportd: tfportqsfp0_0 has no addrs
472026-08-31T16:28:09.070ZINFOtfportd: added new ARP entry entry = 10.151.2.174->42:f5:5b:89:67:7e interface = tfportqsfp0_0
482026-08-31T16:28:09.469ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
492026-08-31T16:28:09.474ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
502026-08-31T16:28:09.477ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
512026-08-31T16:28:09.478ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
522026-08-31T16:28:09.479ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-08-31T16:27:39.570258812Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
532026-08-31T16:28:09.479ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 16760372104655995029 unit = oximeter
542026-08-31T16:28:09.479ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
552026-08-31T16:28:10.045ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
562026-08-31T16:28:10.050ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
572026-08-31T16:28:10.053ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
582026-08-31T16:28:11.034ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
592026-08-31T16:28:11.038ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
602026-08-31T16:28:11.041ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
612026-08-31T16:28:11.930ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-08-31T16:28:11.935ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
632026-08-31T16:28:11.938ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
642026-08-31T16:28:13.298ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-08-31T16:28:13.302ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
662026-08-31T16:28:13.305ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
672026-08-31T16:28:14.254ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-08-31T16:28:14.260ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
692026-08-31T16:28:14.264ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
702026-08-31T16:28:15.504ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-08-31T16:28:15.509ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
722026-08-31T16:28:15.512ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
732026-08-31T16:28:16.655ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
742026-08-31T16:28:17.005ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
752026-08-31T16:28:17.009ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
762026-08-31T16:28:17.012ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
772026-08-31T16:28:18.463ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
782026-08-31T16:28:18.467ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
792026-08-31T16:28:18.471ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
802026-08-31T16:28:19.081ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
812026-08-31T16:28:19.085ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
822026-08-31T16:28:19.088ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
832026-08-31T16:28:19.822ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter
842026-08-31T16:28:20.195ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
852026-08-31T16:28:20.199ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
862026-08-31T16:28:20.202ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
872026-08-31T16:28:21.264ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
882026-08-31T16:28:21.270ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
892026-08-31T16:28:21.273ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
902026-08-31T16:28:21.657ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
912026-08-31T16:28:21.659ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 15.020667062s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
922026-08-31T16:28:21.927ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
932026-08-31T16:28:21.932ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
942026-08-31T16:28:21.935ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
952026-08-31T16:28:22.030ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2de03509-a130-46f9-b202-200f0008608c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("97eb4bac-a1da-4d58-bc01-6d83ce489de8.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 806, tv_nsec: 792978887 } }) unit = producer-server
962026-08-31T16:28:22.031ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
972026-08-31T16:28:22.040ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
982026-08-31T16:28:22.526ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
992026-08-31T16:28:22.531ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1002026-08-31T16:28:22.535ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1012026-08-31T16:28:23.545ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1022026-08-31T16:28:23.550ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1032026-08-31T16:28:23.553ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1042026-08-31T16:28:24.458ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1052026-08-31T16:28:24.463ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1062026-08-31T16:28:24.466ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1072026-08-31T16:28:25.556ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1082026-08-31T16:28:25.562ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1092026-08-31T16:28:25.566ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1102026-08-31T16:28:26.299ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1112026-08-31T16:28:26.304ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1122026-08-31T16:28:26.307ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1132026-08-31T16:28:26.628ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
1142026-08-31T16:28:27.345ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-08-31T16:28:27.350ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1162026-08-31T16:28:27.353ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1172026-08-31T16:28:28.248ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1182026-08-31T16:28:28.252ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1192026-08-31T16:28:28.256ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1202026-08-31T16:28:28.859ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1212026-08-31T16:28:28.864ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1222026-08-31T16:28:28.867ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1232026-08-31T16:28:29.602ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1242026-08-31T16:28:29.606ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1252026-08-31T16:28:29.609ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1262026-08-31T16:28:29.793ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter
1272026-08-31T16:28:30.479ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282026-08-31T16:28:30.484ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1292026-08-31T16:28:30.488ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1302026-08-31T16:28:31.348ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312026-08-31T16:28:31.353ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1322026-08-31T16:28:31.357ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1332026-08-31T16:28:32.651ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342026-08-31T16:28:32.656ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1352026-08-31T16:28:32.660ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1362026-08-31T16:28:33.405ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-08-31T16:28:33.410ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1382026-08-31T16:28:33.412ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1392026-08-31T16:28:34.854ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-08-31T16:28:34.858ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1412026-08-31T16:28:34.861ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1422026-08-31T16:28:35.963ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-08-31T16:28:35.968ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1442026-08-31T16:28:35.971ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1452026-08-31T16:28:36.488ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-08-31T16:28:36.492ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1472026-08-31T16:28:36.495ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1482026-08-31T16:28:36.603ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
1492026-08-31T16:28:36.999ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1502026-08-31T16:28:37.000ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 30.406485325s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1512026-08-31T16:28:37.291ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-08-31T16:28:37.296ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1532026-08-31T16:28:37.299ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1542026-08-31T16:28:37.313ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2de03509-a130-46f9-b202-200f0008608c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("97eb4bac-a1da-4d58-bc01-6d83ce489de8.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 822, tv_nsec: 119718443 } }) unit = producer-server
1552026-08-31T16:28:37.313ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1562026-08-31T16:28:37.320ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1572026-08-31T16:28:37.815ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1582026-08-31T16:28:37.820ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1592026-08-31T16:28:37.824ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1602026-08-31T16:28:38.737ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1612026-08-31T16:28:38.742ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1622026-08-31T16:28:38.745ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1632026-08-31T16:28:39.768ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter
1642026-08-31T16:28:39.947ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1652026-08-31T16:28:39.951ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1662026-08-31T16:28:39.954ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1672026-08-31T16:28:40.781ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1682026-08-31T16:28:40.786ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1692026-08-31T16:28:40.789ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1702026-08-31T16:28:41.984ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1712026-08-31T16:28:41.988ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1722026-08-31T16:28:41.991ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1732026-08-31T16:28:43.336ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1742026-08-31T16:28:43.340ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1752026-08-31T16:28:43.344ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1762026-08-31T16:28:44.126ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1772026-08-31T16:28:44.131ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1782026-08-31T16:28:44.134ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1792026-08-31T16:28:45.065ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1802026-08-31T16:28:45.070ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1812026-08-31T16:28:45.073ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1822026-08-31T16:28:46.303ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1832026-08-31T16:28:46.308ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1842026-08-31T16:28:46.311ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1852026-08-31T16:28:46.577ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
1862026-08-31T16:28:47.782ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1872026-08-31T16:28:47.788ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1882026-08-31T16:28:47.791ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1892026-08-31T16:28:48.442ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1902026-08-31T16:28:48.447ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1912026-08-31T16:28:48.450ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1922026-08-31T16:28:49.718ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1932026-08-31T16:28:49.723ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1942026-08-31T16:28:49.726ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1952026-08-31T16:28:49.742ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter
1962026-08-31T16:28:50.949ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1972026-08-31T16:28:50.955ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
1982026-08-31T16:28:50.958ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1992026-08-31T16:28:51.578ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-08-31T16:28:51.582ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2012026-08-31T16:28:51.586ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2022026-08-31T16:28:52.280ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2032026-08-31T16:28:52.281ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 45.729310824s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2042026-08-31T16:28:52.458ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2052026-08-31T16:28:52.463ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2062026-08-31T16:28:52.466ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2072026-08-31T16:28:52.884ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2de03509-a130-46f9-b202-200f0008608c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("97eb4bac-a1da-4d58-bc01-6d83ce489de8.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 837, tv_nsec: 733825028 } }) unit = producer-server
2082026-08-31T16:28:52.885ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2092026-08-31T16:28:52.892ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2102026-08-31T16:28:53.442ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2112026-08-31T16:28:53.448ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2122026-08-31T16:28:53.451ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2132026-08-31T16:28:54.344ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2142026-08-31T16:28:54.349ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2152026-08-31T16:28:54.353ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2162026-08-31T16:28:55.139ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2172026-08-31T16:28:55.144ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2182026-08-31T16:28:55.147ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2192026-08-31T16:28:56.162ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2202026-08-31T16:28:56.168ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2212026-08-31T16:28:56.171ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2222026-08-31T16:28:56.552ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
2232026-08-31T16:28:57.041ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2242026-08-31T16:28:57.045ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2252026-08-31T16:28:57.048ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2262026-08-31T16:28:57.616ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2272026-08-31T16:28:57.620ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2282026-08-31T16:28:57.623ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2292026-08-31T16:28:58.480ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2302026-08-31T16:28:58.485ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2312026-08-31T16:28:58.488ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2322026-08-31T16:28:59.066ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2332026-08-31T16:28:59.071ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2342026-08-31T16:28:59.074ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2352026-08-31T16:28:59.717ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter
2362026-08-31T16:29:00.482ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372026-08-31T16:29:00.490ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2382026-08-31T16:29:00.493ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2392026-08-31T16:29:01.223ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402026-08-31T16:29:01.235ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2412026-08-31T16:29:01.240ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2422026-08-31T16:29:02.540ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2432026-08-31T16:29:02.552ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2442026-08-31T16:29:02.564ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2452026-08-31T16:29:03.998ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462026-08-31T16:29:04.005ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2472026-08-31T16:29:04.007ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2482026-08-31T16:29:05.427ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2492026-08-31T16:29:05.500ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2502026-08-31T16:29:05.579ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2512026-08-31T16:29:06.527ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
2522026-08-31T16:29:06.646ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2532026-08-31T16:29:06.652ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2542026-08-31T16:29:06.656ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2552026-08-31T16:29:07.852ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2562026-08-31T16:29:07.853ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 61.343689224s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2572026-08-31T16:29:07.876ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2582026-08-31T16:29:07.883ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2592026-08-31T16:29:07.887ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2602026-08-31T16:29:08.508ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2612026-08-31T16:29:08.513ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2622026-08-31T16:29:08.516ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2632026-08-31T16:29:09.101ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2642026-08-31T16:29:09.106ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2652026-08-31T16:29:09.109ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2662026-08-31T16:29:09.126ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2de03509-a130-46f9-b202-200f0008608c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("97eb4bac-a1da-4d58-bc01-6d83ce489de8.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 854, tv_nsec: 19454102 } }) unit = producer-server
2672026-08-31T16:29:09.126ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2682026-08-31T16:29:09.133ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2692026-08-31T16:29:09.692ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter
2702026-08-31T16:29:10.555ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2712026-08-31T16:29:10.562ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2722026-08-31T16:29:10.565ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2732026-08-31T16:29:11.858ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2742026-08-31T16:29:11.863ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2752026-08-31T16:29:11.867ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2762026-08-31T16:29:13.052ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2772026-08-31T16:29:13.057ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2782026-08-31T16:29:13.060ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2792026-08-31T16:29:14.427ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2802026-08-31T16:29:14.432ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2812026-08-31T16:29:14.435ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2822026-08-31T16:29:15.084ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2832026-08-31T16:29:15.089ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2842026-08-31T16:29:15.092ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2852026-08-31T16:29:16.207ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2862026-08-31T16:29:16.212ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2872026-08-31T16:29:16.216ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2882026-08-31T16:29:16.516ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
2892026-08-31T16:29:17.635ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2902026-08-31T16:29:17.640ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2912026-08-31T16:29:17.643ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2922026-08-31T16:29:18.459ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2932026-08-31T16:29:18.464ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2942026-08-31T16:29:18.467ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2952026-08-31T16:29:19.689ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter
2962026-08-31T16:29:19.851ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2972026-08-31T16:29:19.855ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
2982026-08-31T16:29:19.858ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2992026-08-31T16:29:21.345ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3002026-08-31T16:29:21.350ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3012026-08-31T16:29:21.353ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3022026-08-31T16:29:22.408ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3032026-08-31T16:29:22.414ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3042026-08-31T16:29:22.418ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3052026-08-31T16:29:23.473ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3062026-08-31T16:29:23.479ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3072026-08-31T16:29:23.482ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3082026-08-31T16:29:24.129ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3092026-08-31T16:29:24.130ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 77.629379593s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3102026-08-31T16:29:24.975ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3112026-08-31T16:29:24.981ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3122026-08-31T16:29:24.985ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3132026-08-31T16:29:25.882ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3142026-08-31T16:29:25.892ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3152026-08-31T16:29:25.900ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3162026-08-31T16:29:26.545ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
3172026-08-31T16:29:26.746ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2de03509-a130-46f9-b202-200f0008608c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("97eb4bac-a1da-4d58-bc01-6d83ce489de8.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 871, tv_nsec: 645358747 } }) unit = producer-server
3182026-08-31T16:29:26.747ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
3192026-08-31T16:29:26.755ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3202026-08-31T16:29:27.275ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3212026-08-31T16:29:27.281ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3222026-08-31T16:29:27.284ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3232026-08-31T16:29:28.315ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3242026-08-31T16:29:28.321ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3252026-08-31T16:29:28.324ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3262026-08-31T16:29:29.089ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3272026-08-31T16:29:29.108ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3282026-08-31T16:29:29.122ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3292026-08-31T16:29:29.691ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter
3302026-08-31T16:29:30.200ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3312026-08-31T16:29:30.205ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3322026-08-31T16:29:30.208ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3332026-08-31T16:29:30.928ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3342026-08-31T16:29:30.933ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3352026-08-31T16:29:30.937ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3362026-08-31T16:29:32.238ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3372026-08-31T16:29:32.246ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3382026-08-31T16:29:32.251ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3392026-08-31T16:29:33.361ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3402026-08-31T16:29:33.369ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3412026-08-31T16:29:33.373ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3422026-08-31T16:29:34.251ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3432026-08-31T16:29:34.257ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3442026-08-31T16:29:34.261ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3452026-08-31T16:29:35.094ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3462026-08-31T16:29:35.100ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3472026-08-31T16:29:35.105ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3482026-08-31T16:29:36.551ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
3492026-08-31T16:29:36.653ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3502026-08-31T16:29:36.663ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3512026-08-31T16:29:36.669ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3522026-08-31T16:29:40.270ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter
3532026-08-31T16:29:40.928ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3542026-08-31T16:29:41.029ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3552026-08-31T16:29:41.068ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3562026-08-31T16:29:42.202ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3572026-08-31T16:29:42.386ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3582026-08-31T16:29:42.396ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 95.882331113s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3592026-08-31T16:29:42.504ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3602026-08-31T16:29:42.564ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3612026-08-31T16:29:43.630ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3622026-08-31T16:29:43.680ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3632026-08-31T16:29:43.688ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3642026-08-31T16:29:45.581ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3652026-08-31T16:29:45.784ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3662026-08-31T16:29:45.795ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3672026-08-31T16:29:46.419ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3682026-08-31T16:29:46.427ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3692026-08-31T16:29:46.432ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3702026-08-31T16:29:46.573ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
3712026-08-31T16:29:47.592ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3722026-08-31T16:29:47.630ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3732026-08-31T16:29:47.674ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3742026-08-31T16:29:48.516ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3752026-08-31T16:29:48.524ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3762026-08-31T16:29:48.530ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3772026-08-31T16:29:49.879ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3782026-08-31T16:29:49.889ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3792026-08-31T16:29:49.896ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3802026-08-31T16:29:50.286ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter
3812026-08-31T16:29:51.210ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3822026-08-31T16:29:51.218ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3832026-08-31T16:29:51.223ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3842026-08-31T16:29:51.750ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3852026-08-31T16:29:51.765ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3862026-08-31T16:29:51.772ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3872026-08-31T16:29:52.357ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3882026-08-31T16:29:52.397ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3892026-08-31T16:29:52.403ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3902026-08-31T16:29:53.527ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2de03509-a130-46f9-b202-200f0008608c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("97eb4bac-a1da-4d58-bc01-6d83ce489de8.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ed39db1d-f9a4-4481-98a4-d08a7faf8f62.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 898, tv_nsec: 421818938 } }) unit = producer-server
3912026-08-31T16:29:53.529ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
3922026-08-31T16:29:53.540ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3932026-08-31T16:29:53.549ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3942026-08-31T16:29:53.553ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3952026-08-31T16:29:53.562ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3962026-08-31T16:29:54.362ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3972026-08-31T16:29:54.374ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
3982026-08-31T16:29:54.382ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3992026-08-31T16:29:55.521ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4002026-08-31T16:29:55.530ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
4012026-08-31T16:29:55.535ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4022026-08-31T16:29:56.579ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 4613352465068422250 n_samples = 6 unit = oximeter
4032026-08-31T16:29:56.707ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4042026-08-31T16:29:56.717ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
4052026-08-31T16:29:56.723ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4062026-08-31T16:29:57.989ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4072026-08-31T16:29:57.996ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
4082026-08-31T16:29:58.000ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4092026-08-31T16:29:58.622ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4102026-08-31T16:29:58.634ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
4112026-08-31T16:29:58.640ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4122026-08-31T16:29:59.233ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4132026-08-31T16:29:59.243ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
4142026-08-31T16:29:59.248ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4152026-08-31T16:30:00.024ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4162026-08-31T16:30:00.032ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d096:edff:fe57:65ab]
4172026-08-31T16:30:00.038ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4182026-08-31T16:30:00.291ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16760372104655995029 n_samples = 6 unit = oximeter