Max level shown:
1 [ Jul 22 18:12:17 Enabled. ]
2 [ Jul 22 18:12:17 Rereading configuration. ]
3 [ Jul 22 18:12:17 Rereading configuration. ]
4 [ Jul 22 18:12:29 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jul 22 18:12:29 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-07-22T18:12:29.045ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-07-22T18:12:29.047ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
92026-07-22T18:12:29.047ZINFOtfportd: running in sync-only mode
102026-07-22T18:12:29.047ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-07-22T18:12:29.049ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-07-22T18:12:29.051ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 4b9283f7-c425-44ae-a7b0-d28fa982e9e1, ), sled_id: Some( 990c5939-20fa-40a3-ab68-23185a5fd8e3, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-07-22T18:12:29.051ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-07-22T18:12:29.055ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::fd:e4ff:fea4:bf50->02:fd:e4:a4:bf:50 interface = tfportrear0_0
162026-07-22T18:12:29.056ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-07-22T18:12:29.061ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
182026-07-22T18:12:29.067ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Jul 22 18:12:29 Rereading configuration. ]
20 [ Jul 22 18:12:29 Executing refresh method (:kill). ]
21 [ Jul 22 18:12:29 Stopping because all processes in service exited. ]
22 [ Jul 22 18:12:29 Executing stop method (:kill). ]
23 [ Jul 22 18:12:29 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Jul 22 18:12:29 Method "start" exited with status 0. ]
252026-07-22T18:12:29.808ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-07-22T18:12:29.809ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
272026-07-22T18:12:29.809ZINFOtfportd: running in sync-only mode
282026-07-22T18:12:29.810ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-07-22T18:12:29.812ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-07-22T18:12:29.812ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 4b9283f7-c425-44ae-a7b0-d28fa982e9e1, sled_id: 990c5939-20fa-40a3-ab68-23185a5fd8e3, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 5e4eeaad-c952-4f8d-a59e-9bf7aa1932e1, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-07-22T18:12:29.812ZDEBGtfportd: created producer ID producer_id = 8cd03063-2d8d-4b80-a946-6a6059c403e3 unit = oximeter
322026-07-22T18:12:29.813ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 4b9283f7-c425-44ae-a7b0-d28fa982e9e1, ), sled_id: Some( 990c5939-20fa-40a3-ab68-23185a5fd8e3, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
332026-07-22T18:12:29.821ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-07-22T18:12:29.822ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::80bd:94ff:fe21:4113->82:bd:94:21:41:13 interface = tfportrear0_0
352026-07-22T18:12:29.823ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::fd:e4ff:fea4:bf50->02:fd:e4:a4:bf:50 interface = tfportrear0_0
362026-07-22T18:12:29.824ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-07-22T18:12:29.825ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:46276 method = GET path = /{producer_id} unit = producer-server versions = all
382026-07-22T18:12:29.825ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:46276 unit = producer-server
392026-07-22T18:12:29.825ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:46276 unit = producer-server
402026-07-22T18:12:29.825ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:46276 unit = producer-server
412026-07-22T18:12:29.825ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-07-22T18:12:29.826ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-07-22T18:12:29.826ZDEBGtfportd: starting producer registration task unit = producer-server
442026-07-22T18:12:29.826ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:46276 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 8cd03063-2d8d-4b80-a946-6a6059c403e3 unit = producer-server
452026-07-22T18:12:29.826ZINFOtfportd: created oximeter producer server unit = oximeter
462026-07-22T18:12:29.826ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-07-22T18:12:29.827ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
482026-07-22T18:12:29.827ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-07-22T18:12:29.829ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-07-22T18:12:12.809263445Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502026-07-22T18:12:29.829ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 15684839416700532162 unit = oximeter
512026-07-22T18:12:29.829ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522026-07-22T18:12:29.830ZWARNtfportd: tfportqsfp0_0 has no addrs
532026-07-22T18:12:30.462ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
542026-07-22T18:12:30.466ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
552026-07-22T18:12:30.470ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
562026-07-22T18:12:30.470ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
572026-07-22T18:12:30.471ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-07-22T18:12:12.809134672Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
582026-07-22T18:12:30.471ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 9708480280734036341 unit = oximeter
592026-07-22T18:12:30.471ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
602026-07-22T18:12:30.675ZINFOtfportd: added new ARP entry entry = 10.151.1.174->96:d5:f4:b3:0a:b2 interface = tfportqsfp0_0
612026-07-22T18:12:31.513ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-07-22T18:12:31.519ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
632026-07-22T18:12:31.522ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
642026-07-22T18:12:32.595ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-07-22T18:12:32.602ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
662026-07-22T18:12:32.605ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
672026-07-22T18:12:34.028ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-07-22T18:12:34.033ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
692026-07-22T18:12:34.037ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
702026-07-22T18:12:34.828ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a39ac893-e01f-4f85-842d-6f3ecf1f0551.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("acf8ba20-6a4b-45c2-a578-c8b8043f3be6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 679, tv_nsec: 774549156 } }) unit = producer-server
712026-07-22T18:12:34.829ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
722026-07-22T18:12:34.845ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
732026-07-22T18:12:35.135ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-07-22T18:12:35.140ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
752026-07-22T18:12:35.143ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
762026-07-22T18:12:36.097ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-07-22T18:12:36.102ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
782026-07-22T18:12:36.105ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
792026-07-22T18:12:36.995ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-07-22T18:12:37.000ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
812026-07-22T18:12:37.003ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
822026-07-22T18:12:37.601ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-07-22T18:12:37.606ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
842026-07-22T18:12:37.609ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
852026-07-22T18:12:38.625ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862026-07-22T18:12:38.630ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
872026-07-22T18:12:38.634ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
882026-07-22T18:12:39.707ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
892026-07-22T18:12:39.712ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
902026-07-22T18:12:39.715ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
912026-07-22T18:12:39.832ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
922026-07-22T18:12:40.473ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
932026-07-22T18:12:40.991ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942026-07-22T18:12:40.995ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
952026-07-22T18:12:40.998ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
962026-07-22T18:12:42.234ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-07-22T18:12:42.239ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
982026-07-22T18:12:42.242ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
992026-07-22T18:12:43.125ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002026-07-22T18:12:43.130ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1012026-07-22T18:12:43.133ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1022026-07-22T18:12:45.181ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-07-22T18:12:45.187ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1042026-07-22T18:12:45.190ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1052026-07-22T18:12:46.506ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-07-22T18:12:46.510ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1072026-07-22T18:12:46.512ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1082026-07-22T18:12:47.555ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-07-22T18:12:47.560ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1102026-07-22T18:12:47.563ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1112026-07-22T18:12:48.768ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-07-22T18:12:48.773ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1132026-07-22T18:12:48.776ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1142026-07-22T18:12:49.387ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-07-22T18:12:49.392ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1162026-07-22T18:12:49.396ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1172026-07-22T18:12:50.030ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1182026-07-22T18:12:50.035ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1192026-07-22T18:12:50.038ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1202026-07-22T18:12:50.370ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
1212026-07-22T18:12:50.381ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1222026-07-22T18:12:50.382ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 20.020758586s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1232026-07-22T18:12:50.574ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a39ac893-e01f-4f85-842d-6f3ecf1f0551.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("acf8ba20-6a4b-45c2-a578-c8b8043f3be6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 694, tv_nsec: 986203568 } }) unit = producer-server
1242026-07-22T18:12:50.575ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1252026-07-22T18:12:50.583ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1262026-07-22T18:12:50.833ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1272026-07-22T18:12:50.837ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1282026-07-22T18:12:50.841ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1292026-07-22T18:12:51.007ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
1302026-07-22T18:12:51.701ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312026-07-22T18:12:51.706ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1322026-07-22T18:12:51.709ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1332026-07-22T18:12:53.111ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342026-07-22T18:12:53.122ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1352026-07-22T18:12:53.126ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1362026-07-22T18:12:53.976ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-07-22T18:12:53.980ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1382026-07-22T18:12:53.983ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1392026-07-22T18:12:55.163ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-07-22T18:12:55.168ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1412026-07-22T18:12:55.171ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1422026-07-22T18:12:56.676ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-07-22T18:12:56.681ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1442026-07-22T18:12:56.684ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1452026-07-22T18:12:58.035ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-07-22T18:12:58.040ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1472026-07-22T18:12:58.043ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1482026-07-22T18:12:58.620ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-07-22T18:12:58.626ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1502026-07-22T18:12:58.629ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1512026-07-22T18:12:59.276ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-07-22T18:12:59.281ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1532026-07-22T18:12:59.284ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1542026-07-22T18:13:00.020ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1552026-07-22T18:13:00.025ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1562026-07-22T18:13:00.028ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1572026-07-22T18:13:00.345ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
1582026-07-22T18:13:00.726ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1592026-07-22T18:13:00.731ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1602026-07-22T18:13:00.735ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1612026-07-22T18:13:00.982ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
1622026-07-22T18:13:02.132ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1632026-07-22T18:13:02.138ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1642026-07-22T18:13:02.141ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1652026-07-22T18:13:02.672ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1662026-07-22T18:13:02.678ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1672026-07-22T18:13:02.681ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1682026-07-22T18:13:03.954ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1692026-07-22T18:13:03.960ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1702026-07-22T18:13:03.963ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1712026-07-22T18:13:05.305ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-07-22T18:13:05.310ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1732026-07-22T18:13:05.313ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1742026-07-22T18:13:05.544ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1752026-07-22T18:13:05.545ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 35.224449451s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1762026-07-22T18:13:06.247ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a39ac893-e01f-4f85-842d-6f3ecf1f0551.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("acf8ba20-6a4b-45c2-a578-c8b8043f3be6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 710, tv_nsec: 700834304 } }) unit = producer-server
1772026-07-22T18:13:06.247ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1782026-07-22T18:13:06.254ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1792026-07-22T18:13:06.298ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1802026-07-22T18:13:06.303ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1812026-07-22T18:13:06.306ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1822026-07-22T18:13:07.538ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1832026-07-22T18:13:07.543ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1842026-07-22T18:13:07.546ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1852026-07-22T18:13:08.423ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862026-07-22T18:13:08.428ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1872026-07-22T18:13:08.431ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1882026-07-22T18:13:09.626ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-07-22T18:13:09.631ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1902026-07-22T18:13:09.633ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1912026-07-22T18:13:10.320ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
1922026-07-22T18:13:10.957ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
1932026-07-22T18:13:11.077ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1942026-07-22T18:13:11.082ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1952026-07-22T18:13:11.085ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1962026-07-22T18:13:12.257ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1972026-07-22T18:13:12.261ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
1982026-07-22T18:13:12.264ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1992026-07-22T18:13:13.275ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-07-22T18:13:13.279ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2012026-07-22T18:13:13.283ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2022026-07-22T18:13:14.168ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032026-07-22T18:13:14.173ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2042026-07-22T18:13:14.176ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2052026-07-22T18:13:15.186ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2062026-07-22T18:13:15.190ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2072026-07-22T18:13:15.194ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2082026-07-22T18:13:16.496ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-07-22T18:13:16.501ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2102026-07-22T18:13:16.504ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2112026-07-22T18:13:17.643ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-07-22T18:13:17.647ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2132026-07-22T18:13:17.650ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2142026-07-22T18:13:18.783ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2152026-07-22T18:13:18.788ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2162026-07-22T18:13:18.790ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2172026-07-22T18:13:20.148ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2182026-07-22T18:13:20.154ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2192026-07-22T18:13:20.157ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2202026-07-22T18:13:20.294ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
2212026-07-22T18:13:20.932ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
2222026-07-22T18:13:21.214ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2232026-07-22T18:13:21.215ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 50.937875562s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2242026-07-22T18:13:21.357ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2252026-07-22T18:13:21.362ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2262026-07-22T18:13:21.366ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2272026-07-22T18:13:22.585ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a39ac893-e01f-4f85-842d-6f3ecf1f0551.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("acf8ba20-6a4b-45c2-a578-c8b8043f3be6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 727, tv_nsec: 83643786 } }) unit = producer-server
2282026-07-22T18:13:22.586ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
2292026-07-22T18:13:22.592ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
2302026-07-22T18:13:22.827ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2312026-07-22T18:13:22.832ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2322026-07-22T18:13:22.835ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2332026-07-22T18:13:23.574ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-07-22T18:13:23.579ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2352026-07-22T18:13:23.582ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2362026-07-22T18:13:24.336ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372026-07-22T18:13:24.341ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2382026-07-22T18:13:24.344ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2392026-07-22T18:13:24.959ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402026-07-22T18:13:24.964ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2412026-07-22T18:13:24.967ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2422026-07-22T18:13:25.850ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2432026-07-22T18:13:25.856ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2442026-07-22T18:13:25.859ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2452026-07-22T18:13:26.466ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462026-07-22T18:13:26.470ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2472026-07-22T18:13:26.473ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2482026-07-22T18:13:27.316ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2492026-07-22T18:13:27.320ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2502026-07-22T18:13:27.323ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2512026-07-22T18:13:28.574ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2522026-07-22T18:13:28.579ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2532026-07-22T18:13:28.583ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2542026-07-22T18:13:29.722ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2552026-07-22T18:13:29.727ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2562026-07-22T18:13:29.730ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2572026-07-22T18:13:30.271ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
2582026-07-22T18:13:30.605ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2592026-07-22T18:13:30.610ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2602026-07-22T18:13:30.613ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2612026-07-22T18:13:30.907ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
2622026-07-22T18:13:31.411ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2632026-07-22T18:13:31.415ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2642026-07-22T18:13:31.419ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2652026-07-22T18:13:32.423ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2662026-07-22T18:13:32.428ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2672026-07-22T18:13:32.431ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2682026-07-22T18:13:33.472ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692026-07-22T18:13:33.477ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2702026-07-22T18:13:33.480ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2712026-07-22T18:13:34.390ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722026-07-22T18:13:34.395ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2732026-07-22T18:13:34.398ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2742026-07-22T18:13:35.582ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2752026-07-22T18:13:35.587ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2762026-07-22T18:13:35.590ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2772026-07-22T18:13:36.315ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2782026-07-22T18:13:36.320ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2792026-07-22T18:13:36.322ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2802026-07-22T18:13:36.833ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2812026-07-22T18:13:36.837ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2822026-07-22T18:13:36.840ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2832026-07-22T18:13:37.553ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2842026-07-22T18:13:37.554ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 67.320731105s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2852026-07-22T18:13:38.329ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2862026-07-22T18:13:38.334ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2872026-07-22T18:13:38.337ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2882026-07-22T18:13:39.367ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a39ac893-e01f-4f85-842d-6f3ecf1f0551.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("acf8ba20-6a4b-45c2-a578-c8b8043f3be6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 743, tv_nsec: 910616202 } }) unit = producer-server
2892026-07-22T18:13:39.367ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
2902026-07-22T18:13:39.374ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
2912026-07-22T18:13:39.576ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2922026-07-22T18:13:39.580ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2932026-07-22T18:13:39.583ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2942026-07-22T18:13:40.246ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
2952026-07-22T18:13:40.634ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2962026-07-22T18:13:40.639ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
2972026-07-22T18:13:40.643ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2982026-07-22T18:13:40.882ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
2992026-07-22T18:13:41.957ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3002026-07-22T18:13:41.961ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3012026-07-22T18:13:41.963ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3022026-07-22T18:13:42.735ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3032026-07-22T18:13:42.739ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3042026-07-22T18:13:42.742ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3052026-07-22T18:13:44.126ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3062026-07-22T18:13:44.130ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3072026-07-22T18:13:44.133ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3082026-07-22T18:13:45.548ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3092026-07-22T18:13:45.553ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3102026-07-22T18:13:45.557ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3112026-07-22T18:13:46.175ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3122026-07-22T18:13:46.180ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3132026-07-22T18:13:46.183ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3142026-07-22T18:13:46.798ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3152026-07-22T18:13:46.802ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3162026-07-22T18:13:46.805ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3172026-07-22T18:13:48.095ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3182026-07-22T18:13:48.100ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3192026-07-22T18:13:48.103ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3202026-07-22T18:13:48.755ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3212026-07-22T18:13:48.760ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3222026-07-22T18:13:48.763ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3232026-07-22T18:13:50.084ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3242026-07-22T18:13:50.089ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3252026-07-22T18:13:50.093ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3262026-07-22T18:13:50.220ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
3272026-07-22T18:13:50.857ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
3282026-07-22T18:13:51.266ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3292026-07-22T18:13:51.271ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3302026-07-22T18:13:51.274ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3312026-07-22T18:13:51.823ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3322026-07-22T18:13:51.828ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3332026-07-22T18:13:51.831ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3342026-07-22T18:13:52.357ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3352026-07-22T18:13:52.362ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3362026-07-22T18:13:52.366ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3372026-07-22T18:13:53.292ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382026-07-22T18:13:53.297ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3392026-07-22T18:13:53.300ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3402026-07-22T18:13:54.334ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3412026-07-22T18:13:54.335ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 84.147563697s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3422026-07-22T18:13:54.421ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3432026-07-22T18:13:54.425ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3442026-07-22T18:13:54.428ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3452026-07-22T18:13:55.254ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3462026-07-22T18:13:55.259ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3472026-07-22T18:13:55.262ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3482026-07-22T18:13:56.687ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3492026-07-22T18:13:56.692ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3502026-07-22T18:13:56.695ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3512026-07-22T18:13:57.450ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3522026-07-22T18:13:57.455ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3532026-07-22T18:13:57.458ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3542026-07-22T18:13:58.443ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3552026-07-22T18:13:58.449ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3562026-07-22T18:13:58.452ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3572026-07-22T18:13:59.514ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a39ac893-e01f-4f85-842d-6f3ecf1f0551.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("acf8ba20-6a4b-45c2-a578-c8b8043f3be6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 764, tv_nsec: 112605330 } }) unit = producer-server
3582026-07-22T18:13:59.515ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
3592026-07-22T18:13:59.523ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
3602026-07-22T18:13:59.708ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3612026-07-22T18:13:59.713ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3622026-07-22T18:13:59.716ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3632026-07-22T18:14:00.195ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
3642026-07-22T18:14:00.683ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3652026-07-22T18:14:00.688ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3662026-07-22T18:14:00.691ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3672026-07-22T18:14:00.832ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
3682026-07-22T18:14:01.436ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3692026-07-22T18:14:01.441ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3702026-07-22T18:14:01.444ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3712026-07-22T18:14:01.997ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3722026-07-22T18:14:02.002ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3732026-07-22T18:14:02.005ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3742026-07-22T18:14:02.992ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3752026-07-22T18:14:02.997ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3762026-07-22T18:14:03.001ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3772026-07-22T18:14:03.826ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3782026-07-22T18:14:03.832ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3792026-07-22T18:14:03.835ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3802026-07-22T18:14:04.910ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3812026-07-22T18:14:04.914ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3822026-07-22T18:14:04.917ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3832026-07-22T18:14:06.398ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3842026-07-22T18:14:06.403ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3852026-07-22T18:14:06.407ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3862026-07-22T18:14:07.638ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3872026-07-22T18:14:07.643ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3882026-07-22T18:14:07.646ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3892026-07-22T18:14:08.923ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3902026-07-22T18:14:08.928ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3912026-07-22T18:14:08.931ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3922026-07-22T18:14:10.094ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3932026-07-22T18:14:10.099ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3942026-07-22T18:14:10.106ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3952026-07-22T18:14:10.170ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
3962026-07-22T18:14:10.806ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
3972026-07-22T18:14:11.340ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3982026-07-22T18:14:11.345ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
3992026-07-22T18:14:11.349ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4002026-07-22T18:14:11.914ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4012026-07-22T18:14:11.919ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4022026-07-22T18:14:11.922ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4032026-07-22T18:14:12.595ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4042026-07-22T18:14:12.600ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4052026-07-22T18:14:12.603ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4062026-07-22T18:14:14.095ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4072026-07-22T18:14:14.099ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4082026-07-22T18:14:14.102ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4092026-07-22T18:14:14.484ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4102026-07-22T18:14:14.485ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 104.351461688s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
4112026-07-22T18:14:15.462ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4122026-07-22T18:14:15.467ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4132026-07-22T18:14:15.470ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4142026-07-22T18:14:16.247ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4152026-07-22T18:14:16.252ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4162026-07-22T18:14:16.256ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4172026-07-22T18:14:17.547ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4182026-07-22T18:14:17.552ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4192026-07-22T18:14:17.556ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4202026-07-22T18:14:18.661ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4212026-07-22T18:14:18.666ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4222026-07-22T18:14:18.668ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4232026-07-22T18:14:19.871ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4242026-07-22T18:14:19.876ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4252026-07-22T18:14:19.879ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4262026-07-22T18:14:20.145ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
4272026-07-22T18:14:20.781ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
4282026-07-22T18:14:21.128ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4292026-07-22T18:14:21.132ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4302026-07-22T18:14:21.135ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4312026-07-22T18:14:22.000ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4322026-07-22T18:14:22.006ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4332026-07-22T18:14:22.009ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4342026-07-22T18:14:22.618ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4352026-07-22T18:14:22.622ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4362026-07-22T18:14:22.625ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4372026-07-22T18:14:23.443ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4382026-07-22T18:14:23.447ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4392026-07-22T18:14:23.451ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4402026-07-22T18:14:24.478ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a39ac893-e01f-4f85-842d-6f3ecf1f0551.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("acf8ba20-6a4b-45c2-a578-c8b8043f3be6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 789, tv_nsec: 144088623 } }) unit = producer-server
4412026-07-22T18:14:24.479ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
4422026-07-22T18:14:24.485ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
4432026-07-22T18:14:24.602ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4442026-07-22T18:14:24.605ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4452026-07-22T18:14:24.608ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4462026-07-22T18:14:25.960ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4472026-07-22T18:14:25.965ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4482026-07-22T18:14:25.968ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4492026-07-22T18:14:27.413ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4502026-07-22T18:14:27.418ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4512026-07-22T18:14:27.421ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4522026-07-22T18:14:28.277ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4532026-07-22T18:14:28.282ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4542026-07-22T18:14:28.285ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4552026-07-22T18:14:28.927ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4562026-07-22T18:14:28.931ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4572026-07-22T18:14:28.934ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4582026-07-22T18:14:30.021ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4592026-07-22T18:14:30.025ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4602026-07-22T18:14:30.028ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4612026-07-22T18:14:30.120ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
4622026-07-22T18:14:30.755ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
4632026-07-22T18:14:31.064ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4642026-07-22T18:14:31.069ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4652026-07-22T18:14:31.072ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4662026-07-22T18:14:32.355ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4672026-07-22T18:14:32.359ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4682026-07-22T18:14:32.363ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4692026-07-22T18:14:33.249ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4702026-07-22T18:14:33.253ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4712026-07-22T18:14:33.256ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4722026-07-22T18:14:34.214ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4732026-07-22T18:14:34.219ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4742026-07-22T18:14:34.222ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4752026-07-22T18:14:35.166ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4762026-07-22T18:14:35.171ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4772026-07-22T18:14:35.174ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4782026-07-22T18:14:36.267ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4792026-07-22T18:14:36.272ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4802026-07-22T18:14:36.275ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4812026-07-22T18:14:37.556ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4822026-07-22T18:14:37.560ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4832026-07-22T18:14:37.563ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4842026-07-22T18:14:38.765ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4852026-07-22T18:14:38.770ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4862026-07-22T18:14:38.774ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4872026-07-22T18:14:39.447ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4882026-07-22T18:14:39.447ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 129.381838225s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
4892026-07-22T18:14:39.979ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4902026-07-22T18:14:39.984ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4912026-07-22T18:14:39.987ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4922026-07-22T18:14:40.095ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
4932026-07-22T18:14:40.730ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
4942026-07-22T18:14:40.942ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4952026-07-22T18:14:40.947ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4962026-07-22T18:14:40.951ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4972026-07-22T18:14:41.661ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4982026-07-22T18:14:41.667ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
4992026-07-22T18:14:41.670ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5002026-07-22T18:14:43.045ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5012026-07-22T18:14:43.050ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5022026-07-22T18:14:43.053ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5032026-07-22T18:14:43.602ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5042026-07-22T18:14:43.607ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5052026-07-22T18:14:43.610ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5062026-07-22T18:14:44.290ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5072026-07-22T18:14:44.293ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5082026-07-22T18:14:44.297ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5092026-07-22T18:14:45.154ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5102026-07-22T18:14:45.158ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5112026-07-22T18:14:45.162ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5122026-07-22T18:14:45.897ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5132026-07-22T18:14:45.903ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5142026-07-22T18:14:45.905ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5152026-07-22T18:14:47.319ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5162026-07-22T18:14:47.323ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5172026-07-22T18:14:47.326ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5182026-07-22T18:14:47.989ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5192026-07-22T18:14:47.994ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5202026-07-22T18:14:47.997ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5212026-07-22T18:14:48.787ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5222026-07-22T18:14:48.792ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5232026-07-22T18:14:48.795ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5242026-07-22T18:14:49.387ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5252026-07-22T18:14:49.392ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5262026-07-22T18:14:49.396ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5272026-07-22T18:14:50.063ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5282026-07-22T18:14:50.067ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5292026-07-22T18:14:50.070ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
5302026-07-22T18:14:50.070ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5312026-07-22T18:14:50.705ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter
5322026-07-22T18:14:51.119ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5332026-07-22T18:14:51.124ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5342026-07-22T18:14:51.127ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5352026-07-22T18:14:51.715ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5362026-07-22T18:14:51.719ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5372026-07-22T18:14:51.723ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5382026-07-22T18:14:52.732ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5392026-07-22T18:14:52.737ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5402026-07-22T18:14:52.740ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5412026-07-22T18:14:54.018ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5422026-07-22T18:14:54.023ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5432026-07-22T18:14:54.026ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5442026-07-22T18:14:54.814ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5452026-07-22T18:14:54.818ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5462026-07-22T18:14:54.822ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5472026-07-22T18:14:56.125ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5482026-07-22T18:14:56.130ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5492026-07-22T18:14:56.133ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5502026-07-22T18:14:57.158ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5512026-07-22T18:14:57.162ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5522026-07-22T18:14:57.166ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5532026-07-22T18:14:57.373ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a39ac893-e01f-4f85-842d-6f3ecf1f0551.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("acf8ba20-6a4b-45c2-a578-c8b8043f3be6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e0f03cd5-2125-492f-8455-da3edaf4c725.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 822, tv_nsec: 128823152 } }) unit = producer-server
5542026-07-22T18:14:57.374ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
5552026-07-22T18:14:57.381ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
5562026-07-22T18:14:58.152ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5572026-07-22T18:14:58.157ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5582026-07-22T18:14:58.161ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5592026-07-22T18:14:58.892ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5602026-07-22T18:14:58.896ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5612026-07-22T18:14:58.899ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5622026-07-22T18:15:00.045ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15684839416700532162 n_samples = 6 unit = oximeter
5632026-07-22T18:15:00.354ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5642026-07-22T18:15:00.359ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fd:e4ff:fea4:bf50]
5652026-07-22T18:15:00.363ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5662026-07-22T18:15:00.681ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9708480280734036341 n_samples = 6 unit = oximeter