Max level shown:
1 [ Apr 24 17:27:26 Enabled. ]
2 [ Apr 24 17:27:26 Rereading configuration. ]
3 [ Apr 24 17:27:27 Rereading configuration. ]
4 [ Apr 24 17:27:38 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Apr 24 17:27:38 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-04-24T17:27:38.203ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-04-24T17:27:38.205ZINFOtfportd: connected to dpd running softnpu-44a949c9
92026-04-24T17:27:38.205ZINFOtfportd: running in sync-only mode
102026-04-24T17:27:38.206ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-04-24T17:27:38.208ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-04-24T17:27:38.209ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( cd21e5ab-fc11-4e49-bfe1-6c7c4fb85be0, ), sled_id: Some( b26e0000-5eeb-4454-bdcf-d78cdd41c33b, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-04-24T17:27:38.209ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-04-24T17:27:38.213ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::4412:4fff:fe1e:84e6->46:12:4f:1e:84:e6 interface = tfportrear0_0
162026-04-24T17:27:38.214ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::18f9:6ff:fe8a:edc6->1a:f9:06:8a:ed:c6 interface = tfportrear0_0
172026-04-24T17:27:38.215ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
182026-04-24T17:27:38.220ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
192026-04-24T17:27:38.225ZWARNtfportd: tfportqsfp0_0 has no addrs
20 [ Apr 24 17:27:38 Rereading configuration. ]
21 [ Apr 24 17:27:38 Executing refresh method (:kill). ]
22 [ Apr 24 17:27:38 Stopping because all processes in service exited. ]
23 [ Apr 24 17:27:38 Executing stop method (:kill). ]
24 [ Apr 24 17:27:38 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
25 [ Apr 24 17:27:38 Method "start" exited with status 0. ]
262026-04-24T17:27:38.839ZINFOtfportd: connecting to dpd host = [::1] port = 12224
272026-04-24T17:27:38.840ZINFOtfportd: connected to dpd running softnpu-44a949c9
282026-04-24T17:27:38.841ZINFOtfportd: running in sync-only mode
292026-04-24T17:27:38.841ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
302026-04-24T17:27:38.843ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
312026-04-24T17:27:38.843ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: cd21e5ab-fc11-4e49-bfe1-6c7c4fb85be0, sled_id: b26e0000-5eeb-4454-bdcf-d78cdd41c33b, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: f0264351-158b-44df-9f02-389b02c0f8a3, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
322026-04-24T17:27:38.843ZDEBGtfportd: created producer ID producer_id = 8f69a6fa-b771-4ec2-9d2c-836b90396078 unit = oximeter
332026-04-24T17:27:38.845ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( cd21e5ab-fc11-4e49-bfe1-6c7c4fb85be0, ), sled_id: Some( b26e0000-5eeb-4454-bdcf-d78cdd41c33b, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
342026-04-24T17:27:38.851ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
352026-04-24T17:27:38.852ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::4412:4fff:fe1e:84e6->46:12:4f:1e:84:e6 interface = tfportrear0_0
362026-04-24T17:27:38.852ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::18f9:6ff:fe8a:edc6->1a:f9:06:8a:ed:c6 interface = tfportrear0_0
372026-04-24T17:27:38.853ZDEBGtfportd: registered DTrace probes unit = producer-server
382026-04-24T17:27:38.854ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:56254 method = GET path = /{producer_id} unit = producer-server versions = all
392026-04-24T17:27:38.854ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:56254 unit = producer-server
402026-04-24T17:27:38.855ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:56254 unit = producer-server
412026-04-24T17:27:38.855ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:56254 unit = producer-server
422026-04-24T17:27:38.855ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
432026-04-24T17:27:38.855ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
442026-04-24T17:27:38.855ZDEBGtfportd: starting producer registration task unit = producer-server
452026-04-24T17:27:38.855ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:56254 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 8f69a6fa-b771-4ec2-9d2c-836b90396078 unit = producer-server
462026-04-24T17:27:38.855ZINFOtfportd: created oximeter producer server unit = oximeter
472026-04-24T17:27:38.855ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
482026-04-24T17:27:38.856ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
492026-04-24T17:27:38.857ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
502026-04-24T17:27:38.858ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-04-24T17:27:21.988899317Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
512026-04-24T17:27:38.858ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 6068752861234887009 unit = oximeter
522026-04-24T17:27:38.858ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
532026-04-24T17:27:38.859ZWARNtfportd: tfportqsfp0_0 has no addrs
542026-04-24T17:27:39.609ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
552026-04-24T17:27:39.614ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
562026-04-24T17:27:39.617ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
572026-04-24T17:27:39.617ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
582026-04-24T17:27:39.618ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-04-24T17:27:21.988769473Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
592026-04-24T17:27:39.618ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 6408405331313315039 unit = oximeter
602026-04-24T17:27:39.618ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
612026-04-24T17:27:39.894ZINFOtfportd: added new ARP entry entry = 10.151.1.174->06:86:1f:00:56:a5 interface = tfportqsfp0_0
622026-04-24T17:27:40.699ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
632026-04-24T17:27:40.704ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
642026-04-24T17:27:40.707ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
652026-04-24T17:27:41.634ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
662026-04-24T17:27:41.640ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
672026-04-24T17:27:41.643ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
682026-04-24T17:27:42.554ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
692026-04-24T17:27:42.559ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
702026-04-24T17:27:42.562ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
712026-04-24T17:27:43.281ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
722026-04-24T17:27:43.286ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
732026-04-24T17:27:43.289ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
742026-04-24T17:27:43.859ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("46647e07-ece7-4ab7-9ddb-14bc67b060bf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("7c0f82d0-9104-456b-8d63-d9f25c4db110.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 671, tv_nsec: 278977967 } }) unit = producer-server
752026-04-24T17:27:43.859ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
762026-04-24T17:27:43.901ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
772026-04-24T17:27:43.948ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
782026-04-24T17:27:43.953ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
792026-04-24T17:27:43.956ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
802026-04-24T17:27:44.565ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
812026-04-24T17:27:44.570ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
822026-04-24T17:27:44.573ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
832026-04-24T17:27:45.375ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
842026-04-24T17:27:45.380ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
852026-04-24T17:27:45.383ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
862026-04-24T17:27:45.895ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
872026-04-24T17:27:45.901ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
882026-04-24T17:27:45.904ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
892026-04-24T17:27:47.210ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902026-04-24T17:27:47.215ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
912026-04-24T17:27:47.218ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
922026-04-24T17:27:48.555ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
932026-04-24T17:27:48.559ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
942026-04-24T17:27:48.562ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
952026-04-24T17:27:48.863ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
962026-04-24T17:27:49.621ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
972026-04-24T17:27:49.713ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
982026-04-24T17:27:49.717ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
992026-04-24T17:27:49.720ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1002026-04-24T17:27:50.620ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1012026-04-24T17:27:50.625ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1022026-04-24T17:27:50.628ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1032026-04-24T17:27:52.034ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1042026-04-24T17:27:52.039ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1052026-04-24T17:27:52.041ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1062026-04-24T17:27:53.446ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1072026-04-24T17:27:53.452ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1082026-04-24T17:27:53.455ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1092026-04-24T17:27:54.090ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1102026-04-24T17:27:54.095ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1112026-04-24T17:27:54.097ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1122026-04-24T17:27:55.798ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1132026-04-24T17:27:55.804ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1142026-04-24T17:27:55.806ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1152026-04-24T17:27:57.173ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1162026-04-24T17:27:57.179ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1172026-04-24T17:27:57.183ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1182026-04-24T17:27:58.229ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1192026-04-24T17:27:58.235ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1202026-04-24T17:27:58.238ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1212026-04-24T17:27:59.132ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
1222026-04-24T17:27:59.168ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1232026-04-24T17:27:59.169ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 20.047915269s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1242026-04-24T17:27:59.318ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1252026-04-24T17:27:59.324ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1262026-04-24T17:27:59.328ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1272026-04-24T17:27:59.540ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("46647e07-ece7-4ab7-9ddb-14bc67b060bf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("7c0f82d0-9104-456b-8d63-d9f25c4db110.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 686, tv_nsec: 694114942 } }) unit = producer-server
1282026-04-24T17:27:59.540ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1292026-04-24T17:27:59.824ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1302026-04-24T17:27:59.892ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
1312026-04-24T17:28:00.405ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1322026-04-24T17:28:00.419ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1332026-04-24T17:28:00.425ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1342026-04-24T17:28:01.050ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1352026-04-24T17:28:01.057ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1362026-04-24T17:28:01.060ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1372026-04-24T17:28:02.348ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1382026-04-24T17:28:02.357ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1392026-04-24T17:28:02.361ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1402026-04-24T17:28:03.186ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1412026-04-24T17:28:03.192ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1422026-04-24T17:28:03.195ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1432026-04-24T17:28:04.168ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1442026-04-24T17:28:04.173ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1452026-04-24T17:28:04.176ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1462026-04-24T17:28:05.290ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1472026-04-24T17:28:05.294ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1482026-04-24T17:28:05.298ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1492026-04-24T17:28:06.173ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1502026-04-24T17:28:06.178ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1512026-04-24T17:28:06.181ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1522026-04-24T17:28:07.319ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1532026-04-24T17:28:07.326ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1542026-04-24T17:28:07.329ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1552026-04-24T17:28:08.263ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1562026-04-24T17:28:08.268ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1572026-04-24T17:28:08.271ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1582026-04-24T17:28:08.995ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1592026-04-24T17:28:09.000ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1602026-04-24T17:28:09.004ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1612026-04-24T17:28:09.108ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
1622026-04-24T17:28:09.718ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1632026-04-24T17:28:09.726ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1642026-04-24T17:28:09.731ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1652026-04-24T17:28:11.621ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
1662026-04-24T17:28:11.634ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1672026-04-24T17:28:11.642ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1682026-04-24T17:28:11.656ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1692026-04-24T17:28:12.411ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1702026-04-24T17:28:12.419ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1712026-04-24T17:28:12.423ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1722026-04-24T17:28:13.349ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1732026-04-24T17:28:13.360ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1742026-04-24T17:28:13.365ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1752026-04-24T17:28:14.681ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1762026-04-24T17:28:14.688ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1772026-04-24T17:28:14.692ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1782026-04-24T17:28:14.712ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1792026-04-24T17:28:14.757ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 35.678160084s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1802026-04-24T17:28:15.298ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("46647e07-ece7-4ab7-9ddb-14bc67b060bf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("7c0f82d0-9104-456b-8d63-d9f25c4db110.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 702, tv_nsec: 495599111 } }) unit = producer-server
1812026-04-24T17:28:15.299ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1822026-04-24T17:28:15.354ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1832026-04-24T17:28:15.573ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1842026-04-24T17:28:15.581ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1852026-04-24T17:28:15.586ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1862026-04-24T17:28:16.630ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1872026-04-24T17:28:16.634ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1882026-04-24T17:28:16.637ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1892026-04-24T17:28:18.090ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1902026-04-24T17:28:18.097ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1912026-04-24T17:28:18.101ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1922026-04-24T17:28:19.084ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
1932026-04-24T17:28:19.486ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1942026-04-24T17:28:19.492ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1952026-04-24T17:28:19.495ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1962026-04-24T17:28:20.127ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1972026-04-24T17:28:20.132ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
1982026-04-24T17:28:20.135ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1992026-04-24T17:28:20.752ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-04-24T17:28:20.758ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2012026-04-24T17:28:20.762ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2022026-04-24T17:28:21.380ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032026-04-24T17:28:21.385ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2042026-04-24T17:28:21.388ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2052026-04-24T17:28:21.590ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
2062026-04-24T17:28:22.243ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2072026-04-24T17:28:22.252ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2082026-04-24T17:28:22.255ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2092026-04-24T17:28:22.962ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2102026-04-24T17:28:22.966ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2112026-04-24T17:28:22.969ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2122026-04-24T17:28:23.575ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2132026-04-24T17:28:23.580ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2142026-04-24T17:28:23.583ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2152026-04-24T17:28:24.533ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2162026-04-24T17:28:24.538ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2172026-04-24T17:28:24.541ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2182026-04-24T17:28:25.588ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2192026-04-24T17:28:25.593ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2202026-04-24T17:28:25.596ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2212026-04-24T17:28:26.407ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2222026-04-24T17:28:26.413ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2232026-04-24T17:28:26.416ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2242026-04-24T17:28:27.607ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2252026-04-24T17:28:27.612ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2262026-04-24T17:28:27.615ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2272026-04-24T17:28:28.256ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2282026-04-24T17:28:28.262ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2292026-04-24T17:28:28.265ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2302026-04-24T17:28:29.067ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
2312026-04-24T17:28:29.275ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2322026-04-24T17:28:29.280ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2332026-04-24T17:28:29.283ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2342026-04-24T17:28:30.315ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2352026-04-24T17:28:30.316ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 51.279155945s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2362026-04-24T17:28:30.665ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372026-04-24T17:28:30.670ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2382026-04-24T17:28:30.673ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2392026-04-24T17:28:31.565ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
2402026-04-24T17:28:31.583ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("46647e07-ece7-4ab7-9ddb-14bc67b060bf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("7c0f82d0-9104-456b-8d63-d9f25c4db110.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 718, tv_nsec: 824574516 } }) unit = producer-server
2412026-04-24T17:28:31.583ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2422026-04-24T17:28:31.614ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2432026-04-24T17:28:31.907ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2442026-04-24T17:28:31.912ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2452026-04-24T17:28:31.915ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2462026-04-24T17:28:32.455ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2472026-04-24T17:28:32.460ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2482026-04-24T17:28:32.463ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2492026-04-24T17:28:33.332ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2502026-04-24T17:28:33.337ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2512026-04-24T17:28:33.340ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2522026-04-24T17:28:34.473ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2532026-04-24T17:28:34.479ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2542026-04-24T17:28:34.482ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2552026-04-24T17:28:35.713ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2562026-04-24T17:28:35.719ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2572026-04-24T17:28:35.722ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2582026-04-24T17:28:37.227ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2592026-04-24T17:28:37.232ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2602026-04-24T17:28:37.236ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2612026-04-24T17:28:38.185ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2622026-04-24T17:28:38.191ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2632026-04-24T17:28:38.194ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2642026-04-24T17:28:39.063ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
2652026-04-24T17:28:39.530ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2662026-04-24T17:28:39.536ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2672026-04-24T17:28:39.539ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2682026-04-24T17:28:40.497ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692026-04-24T17:28:40.503ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2702026-04-24T17:28:40.513ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2712026-04-24T17:28:41.426ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722026-04-24T17:28:41.432ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2732026-04-24T17:28:41.435ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2742026-04-24T17:28:41.568ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
2752026-04-24T17:28:42.108ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2762026-04-24T17:28:42.114ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2772026-04-24T17:28:42.119ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2782026-04-24T17:28:43.148ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2792026-04-24T17:28:43.168ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2802026-04-24T17:28:43.182ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2812026-04-24T17:28:44.191ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2822026-04-24T17:28:44.199ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2832026-04-24T17:28:44.203ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2842026-04-24T17:28:45.270ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2852026-04-24T17:28:45.277ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2862026-04-24T17:28:45.285ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2872026-04-24T17:28:45.989ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2882026-04-24T17:28:45.996ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2892026-04-24T17:28:46.000ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2902026-04-24T17:28:46.616ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2912026-04-24T17:28:46.623ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 67.588081467s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2922026-04-24T17:28:47.052ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2932026-04-24T17:28:47.068ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2942026-04-24T17:28:47.083ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2952026-04-24T17:28:48.873ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2962026-04-24T17:28:49.028ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
2972026-04-24T17:28:49.047ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2982026-04-24T17:28:51.054ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
2992026-04-24T17:28:51.067ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3002026-04-24T17:28:51.070ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("46647e07-ece7-4ab7-9ddb-14bc67b060bf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("7c0f82d0-9104-456b-8d63-d9f25c4db110.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 738, tv_nsec: 306547918 } }) unit = producer-server
3012026-04-24T17:28:51.071ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
3022026-04-24T17:28:51.259ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3032026-04-24T17:28:51.265ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3042026-04-24T17:28:51.269ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3052026-04-24T17:28:51.572ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
3062026-04-24T17:28:52.434ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3072026-04-24T17:28:52.442ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3082026-04-24T17:28:52.447ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3092026-04-24T17:28:53.596ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3102026-04-24T17:28:53.605ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3112026-04-24T17:28:53.609ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3122026-04-24T17:28:54.832ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3132026-04-24T17:28:54.838ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3142026-04-24T17:28:54.842ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3152026-04-24T17:28:55.817ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3162026-04-24T17:28:55.826ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3172026-04-24T17:28:55.830ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3182026-04-24T17:28:56.956ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3192026-04-24T17:28:56.964ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3202026-04-24T17:28:56.969ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3212026-04-24T17:28:57.780ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3222026-04-24T17:28:57.786ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3232026-04-24T17:28:57.789ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3242026-04-24T17:28:59.156ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3252026-04-24T17:28:59.161ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3262026-04-24T17:28:59.164ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3272026-04-24T17:29:00.643ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3282026-04-24T17:29:00.647ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3292026-04-24T17:29:00.650ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3302026-04-24T17:29:01.057ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
3312026-04-24T17:29:01.574ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
3322026-04-24T17:29:02.056ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3332026-04-24T17:29:02.062ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3342026-04-24T17:29:02.065ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3352026-04-24T17:29:03.305ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3362026-04-24T17:29:03.310ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3372026-04-24T17:29:03.313ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3382026-04-24T17:29:04.688ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3392026-04-24T17:29:04.694ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3402026-04-24T17:29:04.701ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3412026-04-24T17:29:05.836ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3422026-04-24T17:29:05.841ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3432026-04-24T17:29:05.844ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3442026-04-24T17:29:06.261ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3452026-04-24T17:29:06.263ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 87.228743696s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3462026-04-24T17:29:06.441ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3472026-04-24T17:29:06.446ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3482026-04-24T17:29:06.449ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3492026-04-24T17:29:07.612ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3502026-04-24T17:29:07.619ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3512026-04-24T17:29:07.623ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3522026-04-24T17:29:08.333ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3532026-04-24T17:29:08.339ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3542026-04-24T17:29:08.342ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3552026-04-24T17:29:09.129ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3562026-04-24T17:29:09.134ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3572026-04-24T17:29:09.137ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3582026-04-24T17:29:09.921ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3592026-04-24T17:29:09.927ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3602026-04-24T17:29:09.930ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3612026-04-24T17:29:10.022ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("46647e07-ece7-4ab7-9ddb-14bc67b060bf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("7c0f82d0-9104-456b-8d63-d9f25c4db110.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 757, tv_nsec: 263897670 } }) unit = producer-server
3622026-04-24T17:29:10.023ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
3632026-04-24T17:29:10.061ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3642026-04-24T17:29:11.116ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
3652026-04-24T17:29:11.170ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3662026-04-24T17:29:11.175ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3672026-04-24T17:29:11.178ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3682026-04-24T17:29:11.576ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
3692026-04-24T17:29:11.836ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3702026-04-24T17:29:11.841ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3712026-04-24T17:29:11.844ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3722026-04-24T17:29:12.436ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3732026-04-24T17:29:12.442ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3742026-04-24T17:29:12.445ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3752026-04-24T17:29:13.150ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3762026-04-24T17:29:13.156ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3772026-04-24T17:29:13.158ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3782026-04-24T17:29:13.734ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3792026-04-24T17:29:13.740ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3802026-04-24T17:29:13.744ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3812026-04-24T17:29:15.218ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3822026-04-24T17:29:15.225ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3832026-04-24T17:29:15.229ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3842026-04-24T17:29:16.658ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3852026-04-24T17:29:16.664ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3862026-04-24T17:29:16.669ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3872026-04-24T17:29:17.961ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3882026-04-24T17:29:17.968ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3892026-04-24T17:29:17.972ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3902026-04-24T17:29:18.632ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3912026-04-24T17:29:18.639ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3922026-04-24T17:29:18.642ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3932026-04-24T17:29:19.194ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3942026-04-24T17:29:19.199ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3952026-04-24T17:29:19.203ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3962026-04-24T17:29:20.713ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3972026-04-24T17:29:20.722ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
3982026-04-24T17:29:20.727ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3992026-04-24T17:29:21.120ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
4002026-04-24T17:29:21.581ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
4012026-04-24T17:29:22.031ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4022026-04-24T17:29:22.040ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4032026-04-24T17:29:22.043ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4042026-04-24T17:29:23.196ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4052026-04-24T17:29:23.203ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4062026-04-24T17:29:23.206ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4072026-04-24T17:29:23.884ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4082026-04-24T17:29:23.890ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4092026-04-24T17:29:23.893ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4102026-04-24T17:29:24.434ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4112026-04-24T17:29:24.439ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4122026-04-24T17:29:24.442ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4132026-04-24T17:29:25.063ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4142026-04-24T17:29:25.064ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 106.029813544s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
4152026-04-24T17:29:25.482ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4162026-04-24T17:29:25.487ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4172026-04-24T17:29:25.489ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4182026-04-24T17:29:26.566ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4192026-04-24T17:29:26.573ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4202026-04-24T17:29:26.576ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4212026-04-24T17:29:27.223ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4222026-04-24T17:29:27.229ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4232026-04-24T17:29:27.233ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4242026-04-24T17:29:27.934ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4252026-04-24T17:29:27.941ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4262026-04-24T17:29:27.945ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4272026-04-24T17:29:29.115ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4282026-04-24T17:29:29.120ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4292026-04-24T17:29:29.123ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4302026-04-24T17:29:29.699ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4312026-04-24T17:29:29.705ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4322026-04-24T17:29:29.709ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4332026-04-24T17:29:30.870ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4342026-04-24T17:29:30.876ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4352026-04-24T17:29:30.880ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4362026-04-24T17:29:31.124ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
4372026-04-24T17:29:31.498ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4382026-04-24T17:29:31.505ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4392026-04-24T17:29:31.508ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4402026-04-24T17:29:31.583ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
4412026-04-24T17:29:32.184ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4422026-04-24T17:29:32.191ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4432026-04-24T17:29:32.195ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4442026-04-24T17:29:33.539ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4452026-04-24T17:29:33.545ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4462026-04-24T17:29:33.548ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4472026-04-24T17:29:33.623ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("46647e07-ece7-4ab7-9ddb-14bc67b060bf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("7c0f82d0-9104-456b-8d63-d9f25c4db110.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 780, tv_nsec: 865457054 } }) unit = producer-server
4482026-04-24T17:29:33.624ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
4492026-04-24T17:29:33.656ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
4502026-04-24T17:29:33.759ZDEBGtfportd (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "bcdff49f-6efa-4f18-a019-ffb1287b0209", "content-length": "133", "date": "Fri, 24 Apr 2026 17:29:33 GMT"} }) unit = producer-server
4512026-04-24T17:29:33.763ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 114.72889375s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"bcdff49f-6efa-4f18-a019-ffb1287b0209\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 24 Apr 2026 17:29:33 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"bcdff49f-6efa-4f18-a019-ffb1287b0209\\" }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
4522026-04-24T17:29:34.368ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4532026-04-24T17:29:34.374ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4542026-04-24T17:29:34.377ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4552026-04-24T17:29:34.931ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4562026-04-24T17:29:34.937ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4572026-04-24T17:29:34.941ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4582026-04-24T17:29:35.922ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4592026-04-24T17:29:35.929ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4602026-04-24T17:29:35.933ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4612026-04-24T17:29:37.202ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4622026-04-24T17:29:37.207ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4632026-04-24T17:29:37.210ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4642026-04-24T17:29:37.991ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4652026-04-24T17:29:37.997ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4662026-04-24T17:29:38.000ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4672026-04-24T17:29:38.695ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4682026-04-24T17:29:38.701ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4692026-04-24T17:29:38.705ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4702026-04-24T17:29:39.655ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4712026-04-24T17:29:39.661ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4722026-04-24T17:29:39.664ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4732026-04-24T17:29:40.590ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4742026-04-24T17:29:40.596ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4752026-04-24T17:29:40.599ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4762026-04-24T17:29:41.125ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
4772026-04-24T17:29:41.249ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4782026-04-24T17:29:41.255ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4792026-04-24T17:29:41.259ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4802026-04-24T17:29:41.585ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
4812026-04-24T17:29:42.114ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4822026-04-24T17:29:42.120ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4832026-04-24T17:29:42.123ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4842026-04-24T17:29:42.753ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4852026-04-24T17:29:42.760ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4862026-04-24T17:29:42.763ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4872026-04-24T17:29:43.352ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4882026-04-24T17:29:43.358ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4892026-04-24T17:29:43.362ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4902026-04-24T17:29:44.821ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4912026-04-24T17:29:44.826ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4922026-04-24T17:29:44.829ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4932026-04-24T17:29:46.321ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4942026-04-24T17:29:46.326ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4952026-04-24T17:29:46.330ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4962026-04-24T17:29:47.170ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4972026-04-24T17:29:47.175ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
4982026-04-24T17:29:47.184ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4992026-04-24T17:29:48.054ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5002026-04-24T17:29:48.060ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5012026-04-24T17:29:48.063ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5022026-04-24T17:29:49.100ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5032026-04-24T17:29:49.106ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5042026-04-24T17:29:49.109ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5052026-04-24T17:29:50.111ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5062026-04-24T17:29:50.116ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5072026-04-24T17:29:50.119ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5082026-04-24T17:29:51.126ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
5092026-04-24T17:29:51.129ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5102026-04-24T17:29:51.135ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5112026-04-24T17:29:51.138ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5122026-04-24T17:29:51.586ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6408405331313315039 n_samples = 6 unit = oximeter
5132026-04-24T17:29:51.764ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5142026-04-24T17:29:51.769ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5152026-04-24T17:29:51.773ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5162026-04-24T17:29:52.081ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("46647e07-ece7-4ab7-9ddb-14bc67b060bf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("7c0f82d0-9104-456b-8d63-d9f25c4db110.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d97212d8-4e64-4404-bea9-5b642fe25e69.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 799, tv_nsec: 322980122 } }) unit = producer-server
5172026-04-24T17:29:52.082ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
5182026-04-24T17:29:52.113ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
5192026-04-24T17:29:52.170ZDEBGtfportd (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "453d2e2e-9264-4b59-8d95-5edcd6f2c1fe", "content-length": "41", "date": "Fri, 24 Apr 2026 17:29:52 GMT"} }) unit = producer-server
5202026-04-24T17:29:52.171ZDEBGtfportd (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s unit = producer-server wait_period = 150s
5212026-04-24T17:29:52.635ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5222026-04-24T17:29:52.640ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5232026-04-24T17:29:52.643ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5242026-04-24T17:29:53.181ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5252026-04-24T17:29:53.186ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5262026-04-24T17:29:53.189ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5272026-04-24T17:29:54.644ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5282026-04-24T17:29:54.649ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5292026-04-24T17:29:54.653ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5302026-04-24T17:29:55.980ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5312026-04-24T17:29:55.986ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5322026-04-24T17:29:55.990ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5332026-04-24T17:29:56.754ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5342026-04-24T17:29:56.760ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5352026-04-24T17:29:56.763ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5362026-04-24T17:29:57.892ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5372026-04-24T17:29:57.902ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5382026-04-24T17:29:57.906ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5392026-04-24T17:29:59.267ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5402026-04-24T17:29:59.273ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5412026-04-24T17:29:59.277ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5422026-04-24T17:30:00.287ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5432026-04-24T17:30:00.300ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5442026-04-24T17:30:00.304ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5452026-04-24T17:30:01.130ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6068752861234887009 n_samples = 6 unit = oximeter
5462026-04-24T17:30:01.146ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5472026-04-24T17:30:01.156ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4412:4fff:fe1e:84e6]
5482026-04-24T17:30:01.162ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]