Max level shown:
1 [ May 16 04:27:20 Enabled. ]
2 [ May 16 04:27:20 Rereading configuration. ]
3 [ May 16 04:27:21 Rereading configuration. ]
4 [ May 16 04:27:32 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ May 16 04:27:32 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-05-16T04:27:32.626ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-05-16T04:27:32.628ZINFOtfportd: connected to dpd running softnpu-187aee7d
92026-05-16T04:27:32.628ZINFOtfportd: running in sync-only mode
102026-05-16T04:27:32.628ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-05-16T04:27:32.630ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-05-16T04:27:32.637ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( cdcd6d78-8882-4361-bf79-11e2afc3f63f, ), sled_id: Some( dca1fa3b-4844-4709-9635-47575ee8b3ec, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
142026-05-16T04:27:32.637ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-05-16T04:27:32.640ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::70c0:71ff:fec4:c764->72:c0:71:c4:c7:64 interface = tfportrear0_0
162026-05-16T04:27:32.643ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-05-16T04:27:32.651ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
182026-05-16T04:27:32.657ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ May 16 04:27:33 Rereading configuration. ]
20 [ May 16 04:27:33 Executing refresh method (:kill). ]
21 [ May 16 04:27:33 Stopping because all processes in service exited. ]
22 [ May 16 04:27:33 Executing stop method (:kill). ]
23 [ May 16 04:27:33 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ May 16 04:27:33 Method "start" exited with status 0. ]
252026-05-16T04:27:33.391ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-05-16T04:27:33.392ZINFOtfportd: connected to dpd running softnpu-187aee7d
272026-05-16T04:27:33.393ZINFOtfportd: running in sync-only mode
282026-05-16T04:27:33.393ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-05-16T04:27:33.395ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-05-16T04:27:33.395ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: cdcd6d78-8882-4361-bf79-11e2afc3f63f, sled_id: dca1fa3b-4844-4709-9635-47575ee8b3ec, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: b784867f-ca84-4a35-a426-9a3def137f22, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-05-16T04:27:33.395ZDEBGtfportd: created producer ID producer_id = a6e8113c-0a42-4e5d-8d6a-cc881e383af1 unit = oximeter
322026-05-16T04:27:33.396ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( cdcd6d78-8882-4361-bf79-11e2afc3f63f, ), sled_id: Some( dca1fa3b-4844-4709-9635-47575ee8b3ec, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
332026-05-16T04:27:33.403ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-05-16T04:27:33.405ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::70c0:71ff:fec4:c764->72:c0:71:c4:c7:64 interface = tfportrear0_0
352026-05-16T04:27:33.406ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::388c:ff:febc:3b43->3a:8c:00:bc:3b:43 interface = tfportrear0_0
362026-05-16T04:27:33.407ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-05-16T04:27:33.408ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:47695 method = GET path = /{producer_id} unit = producer-server versions = all
382026-05-16T04:27:33.408ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:47695 unit = producer-server
392026-05-16T04:27:33.408ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:47695 unit = producer-server
402026-05-16T04:27:33.408ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:47695 unit = producer-server
412026-05-16T04:27:33.408ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-05-16T04:27:33.409ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-05-16T04:27:33.409ZDEBGtfportd: starting producer registration task unit = producer-server
442026-05-16T04:27:33.409ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:47695 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = a6e8113c-0a42-4e5d-8d6a-cc881e383af1 unit = producer-server
452026-05-16T04:27:33.409ZINFOtfportd: created oximeter producer server unit = oximeter
462026-05-16T04:27:33.409ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-05-16T04:27:33.410ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
482026-05-16T04:27:33.410ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-05-16T04:27:33.411ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-05-16T04:27:16.454910070Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502026-05-16T04:27:33.412ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 11490380268656351138 unit = oximeter
512026-05-16T04:27:33.412ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522026-05-16T04:27:33.413ZWARNtfportd: tfportqsfp0_0 has no addrs
532026-05-16T04:27:33.990ZINFOtfportd: added new ARP entry entry = 10.151.2.174->f6:e6:cf:64:d2:b1 interface = tfportqsfp0_0
542026-05-16T04:27:34.701ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
552026-05-16T04:27:34.706ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
562026-05-16T04:27:34.709ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
572026-05-16T04:27:34.709ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
582026-05-16T04:27:34.710ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-05-16T04:27:16.454777327Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
592026-05-16T04:27:34.710ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 3444092742692835361 unit = oximeter
602026-05-16T04:27:34.710ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
612026-05-16T04:27:36.174ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-05-16T04:27:36.180ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
632026-05-16T04:27:36.184ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
642026-05-16T04:27:37.465ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-05-16T04:27:37.470ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
662026-05-16T04:27:37.473ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
672026-05-16T04:27:38.412ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("957682b6-3ca6-4e84-b11a-7dd8a8366649.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d1f6092c-34a6-4a6a-a2fe-73152fd5c29e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 669, tv_nsec: 460334900 } }) unit = producer-server
682026-05-16T04:27:38.413ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
692026-05-16T04:27:38.429ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
702026-05-16T04:27:38.962ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-05-16T04:27:38.967ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
722026-05-16T04:27:38.970ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
732026-05-16T04:27:39.918ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-05-16T04:27:39.923ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
752026-05-16T04:27:39.926ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
762026-05-16T04:27:41.052ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-05-16T04:27:41.058ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
782026-05-16T04:27:41.061ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
792026-05-16T04:27:42.505ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-05-16T04:27:42.510ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
812026-05-16T04:27:42.513ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
822026-05-16T04:27:43.415ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
832026-05-16T04:27:43.455ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
842026-05-16T04:27:43.459ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
852026-05-16T04:27:43.462ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
862026-05-16T04:27:44.396ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
872026-05-16T04:27:44.401ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
882026-05-16T04:27:44.404ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
892026-05-16T04:27:44.711ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
902026-05-16T04:27:45.797ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
912026-05-16T04:27:45.802ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
922026-05-16T04:27:45.805ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
932026-05-16T04:27:46.720ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942026-05-16T04:27:46.725ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
952026-05-16T04:27:46.728ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
962026-05-16T04:27:47.356ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-05-16T04:27:47.361ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
982026-05-16T04:27:47.364ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
992026-05-16T04:27:49.209ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002026-05-16T04:27:49.214ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1012026-05-16T04:27:49.217ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1022026-05-16T04:27:50.055ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-05-16T04:27:50.061ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1042026-05-16T04:27:50.064ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1052026-05-16T04:27:51.220ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-05-16T04:27:51.225ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1072026-05-16T04:27:51.228ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1082026-05-16T04:27:52.355ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-05-16T04:27:52.360ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1102026-05-16T04:27:52.364ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1112026-05-16T04:27:53.422ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
1122026-05-16T04:27:53.435ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1132026-05-16T04:27:53.436ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 20.022441325s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1142026-05-16T04:27:53.529ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-05-16T04:27:53.534ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1162026-05-16T04:27:53.537ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1172026-05-16T04:27:53.709ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("957682b6-3ca6-4e84-b11a-7dd8a8366649.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d1f6092c-34a6-4a6a-a2fe-73152fd5c29e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 684, tv_nsec: 752087653 } }) unit = producer-server
1182026-05-16T04:27:53.710ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
1192026-05-16T04:27:53.717ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1202026-05-16T04:27:54.721ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
1212026-05-16T04:27:55.007ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1222026-05-16T04:27:55.012ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1232026-05-16T04:27:55.015ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1242026-05-16T04:27:55.595ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1252026-05-16T04:27:55.600ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1262026-05-16T04:27:55.604ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1272026-05-16T04:27:56.875ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282026-05-16T04:27:56.880ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1292026-05-16T04:27:56.883ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1302026-05-16T04:27:58.173ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312026-05-16T04:27:58.179ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1322026-05-16T04:27:58.182ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1332026-05-16T04:27:58.765ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342026-05-16T04:27:58.770ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1352026-05-16T04:27:58.773ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1362026-05-16T04:27:59.549ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-05-16T04:27:59.553ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1382026-05-16T04:27:59.556ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1392026-05-16T04:28:00.644ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-05-16T04:28:00.649ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1412026-05-16T04:28:00.652ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1422026-05-16T04:28:01.764ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-05-16T04:28:01.768ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1442026-05-16T04:28:01.771ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1452026-05-16T04:28:03.115ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-05-16T04:28:03.120ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1472026-05-16T04:28:03.122ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1482026-05-16T04:28:03.450ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
1492026-05-16T04:28:03.813ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1502026-05-16T04:28:03.818ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1512026-05-16T04:28:03.820ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1522026-05-16T04:28:04.750ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
1532026-05-16T04:28:05.227ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1542026-05-16T04:28:05.232ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1552026-05-16T04:28:05.235ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1562026-05-16T04:28:06.229ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1572026-05-16T04:28:06.233ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1582026-05-16T04:28:06.236ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1592026-05-16T04:28:07.074ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1602026-05-16T04:28:07.079ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1612026-05-16T04:28:07.082ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1622026-05-16T04:28:08.008ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1632026-05-16T04:28:08.013ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1642026-05-16T04:28:08.017ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1652026-05-16T04:28:08.759ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1662026-05-16T04:28:08.760ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 35.305362919s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1672026-05-16T04:28:08.797ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1682026-05-16T04:28:08.802ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1692026-05-16T04:28:08.805ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1702026-05-16T04:28:09.294ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("957682b6-3ca6-4e84-b11a-7dd8a8366649.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d1f6092c-34a6-4a6a-a2fe-73152fd5c29e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 700, tv_nsec: 294758391 } }) unit = producer-server
1712026-05-16T04:28:09.294ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
1722026-05-16T04:28:09.301ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1732026-05-16T04:28:10.202ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1742026-05-16T04:28:10.207ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1752026-05-16T04:28:10.211ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1762026-05-16T04:28:11.658ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1772026-05-16T04:28:11.663ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1782026-05-16T04:28:11.666ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1792026-05-16T04:28:13.079ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1802026-05-16T04:28:13.084ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1812026-05-16T04:28:13.088ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1822026-05-16T04:28:13.479ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
1832026-05-16T04:28:13.937ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1842026-05-16T04:28:13.942ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1852026-05-16T04:28:13.945ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1862026-05-16T04:28:14.778ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
1872026-05-16T04:28:14.925ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1882026-05-16T04:28:14.930ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1892026-05-16T04:28:14.933ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1902026-05-16T04:28:15.538ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1912026-05-16T04:28:15.543ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1922026-05-16T04:28:15.546ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1932026-05-16T04:28:16.057ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1942026-05-16T04:28:16.062ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1952026-05-16T04:28:16.066ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1962026-05-16T04:28:17.079ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1972026-05-16T04:28:17.084ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
1982026-05-16T04:28:17.087ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1992026-05-16T04:28:18.197ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-05-16T04:28:18.202ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2012026-05-16T04:28:18.206ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2022026-05-16T04:28:19.631ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032026-05-16T04:28:19.635ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2042026-05-16T04:28:19.638ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2052026-05-16T04:28:20.436ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2062026-05-16T04:28:20.440ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2072026-05-16T04:28:20.443ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2082026-05-16T04:28:21.537ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-05-16T04:28:21.541ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2102026-05-16T04:28:21.544ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2112026-05-16T04:28:22.709ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-05-16T04:28:22.714ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2132026-05-16T04:28:22.717ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2142026-05-16T04:28:23.508ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
2152026-05-16T04:28:24.119ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2162026-05-16T04:28:24.125ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2172026-05-16T04:28:24.128ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2182026-05-16T04:28:24.343ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2192026-05-16T04:28:24.345ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 50.847951086s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2202026-05-16T04:28:24.807ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
2212026-05-16T04:28:25.060ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("957682b6-3ca6-4e84-b11a-7dd8a8366649.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d1f6092c-34a6-4a6a-a2fe-73152fd5c29e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 716, tv_nsec: 18523276 } }) unit = producer-server
2222026-05-16T04:28:25.066ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
2232026-05-16T04:28:25.072ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
2242026-05-16T04:28:25.250ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2252026-05-16T04:28:25.255ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2262026-05-16T04:28:25.258ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2272026-05-16T04:28:25.922ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2282026-05-16T04:28:25.927ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2292026-05-16T04:28:25.930ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2302026-05-16T04:28:26.478ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2312026-05-16T04:28:26.482ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2322026-05-16T04:28:26.485ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2332026-05-16T04:28:27.131ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-05-16T04:28:27.137ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2352026-05-16T04:28:27.139ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2362026-05-16T04:28:28.405ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372026-05-16T04:28:28.409ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2382026-05-16T04:28:28.413ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2392026-05-16T04:28:29.714ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402026-05-16T04:28:29.719ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2412026-05-16T04:28:29.722ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2422026-05-16T04:28:31.159ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2432026-05-16T04:28:31.163ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2442026-05-16T04:28:31.166ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2452026-05-16T04:28:32.536ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462026-05-16T04:28:32.541ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2472026-05-16T04:28:32.545ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2482026-05-16T04:28:33.538ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
2492026-05-16T04:28:33.593ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2502026-05-16T04:28:33.603ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2512026-05-16T04:28:33.605ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2522026-05-16T04:28:34.836ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
2532026-05-16T04:28:34.904ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2542026-05-16T04:28:34.910ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2552026-05-16T04:28:34.913ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2562026-05-16T04:28:36.278ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2572026-05-16T04:28:36.282ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2582026-05-16T04:28:36.285ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2592026-05-16T04:28:37.465ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2602026-05-16T04:28:37.470ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2612026-05-16T04:28:37.472ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2622026-05-16T04:28:38.844ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2632026-05-16T04:28:38.849ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2642026-05-16T04:28:38.852ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2652026-05-16T04:28:40.115ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2662026-05-16T04:28:40.116ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 66.576340935s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2672026-05-16T04:28:40.147ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2682026-05-16T04:28:40.152ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2692026-05-16T04:28:40.155ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2702026-05-16T04:28:40.939ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2712026-05-16T04:28:40.943ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2722026-05-16T04:28:40.946ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2732026-05-16T04:28:41.287ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("957682b6-3ca6-4e84-b11a-7dd8a8366649.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d1f6092c-34a6-4a6a-a2fe-73152fd5c29e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 732, tv_nsec: 201585199 } }) unit = producer-server
2742026-05-16T04:28:41.288ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
2752026-05-16T04:28:41.296ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
2762026-05-16T04:28:42.171ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2772026-05-16T04:28:42.176ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2782026-05-16T04:28:42.179ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2792026-05-16T04:28:43.567ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
2802026-05-16T04:28:43.641ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2812026-05-16T04:28:43.646ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2822026-05-16T04:28:43.650ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2832026-05-16T04:28:44.666ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2842026-05-16T04:28:44.671ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2852026-05-16T04:28:44.675ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2862026-05-16T04:28:44.865ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
2872026-05-16T04:28:45.661ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2882026-05-16T04:28:45.666ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2892026-05-16T04:28:45.670ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2902026-05-16T04:28:46.345ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2912026-05-16T04:28:46.351ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2922026-05-16T04:28:46.353ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2932026-05-16T04:28:47.087ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2942026-05-16T04:28:47.091ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2952026-05-16T04:28:47.094ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2962026-05-16T04:28:47.701ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2972026-05-16T04:28:47.706ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
2982026-05-16T04:28:47.709ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2992026-05-16T04:28:48.450ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3002026-05-16T04:28:48.455ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3012026-05-16T04:28:48.458ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3022026-05-16T04:28:49.329ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3032026-05-16T04:28:49.333ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3042026-05-16T04:28:49.337ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3052026-05-16T04:28:50.765ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3062026-05-16T04:28:50.771ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3072026-05-16T04:28:50.774ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3082026-05-16T04:28:51.607ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3092026-05-16T04:28:51.611ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3102026-05-16T04:28:51.614ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3112026-05-16T04:28:53.067ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3122026-05-16T04:28:53.071ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3132026-05-16T04:28:53.074ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3142026-05-16T04:28:53.596ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
3152026-05-16T04:28:53.914ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3162026-05-16T04:28:53.919ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3172026-05-16T04:28:53.922ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3182026-05-16T04:28:54.711ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3192026-05-16T04:28:54.716ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3202026-05-16T04:28:54.719ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3212026-05-16T04:28:54.894ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
3222026-05-16T04:28:55.269ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3232026-05-16T04:28:55.274ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3242026-05-16T04:28:55.277ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3252026-05-16T04:28:56.337ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3262026-05-16T04:28:56.338ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 82.754622029s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3272026-05-16T04:28:56.472ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3282026-05-16T04:28:56.476ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3292026-05-16T04:28:56.479ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3302026-05-16T04:28:57.746ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3312026-05-16T04:28:57.751ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3322026-05-16T04:28:57.754ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3332026-05-16T04:28:58.816ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3342026-05-16T04:28:58.822ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3352026-05-16T04:28:58.825ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3362026-05-16T04:28:59.996ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3372026-05-16T04:29:00.000ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3382026-05-16T04:29:00.003ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3392026-05-16T04:29:00.820ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3402026-05-16T04:29:00.825ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3412026-05-16T04:29:00.828ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3422026-05-16T04:29:01.296ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("957682b6-3ca6-4e84-b11a-7dd8a8366649.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d1f6092c-34a6-4a6a-a2fe-73152fd5c29e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 752, tv_nsec: 157431205 } }) unit = producer-server
3432026-05-16T04:29:01.297ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
3442026-05-16T04:29:01.305ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
3452026-05-16T04:29:01.900ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3462026-05-16T04:29:01.905ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3472026-05-16T04:29:01.908ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3482026-05-16T04:29:02.670ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3492026-05-16T04:29:02.675ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3502026-05-16T04:29:02.678ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3512026-05-16T04:29:03.425ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3522026-05-16T04:29:03.430ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3532026-05-16T04:29:03.433ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3542026-05-16T04:29:03.625ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
3552026-05-16T04:29:04.387ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3562026-05-16T04:29:04.391ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3572026-05-16T04:29:04.394ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3582026-05-16T04:29:04.923ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
3592026-05-16T04:29:05.476ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3602026-05-16T04:29:05.480ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3612026-05-16T04:29:05.484ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3622026-05-16T04:29:06.939ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3632026-05-16T04:29:06.944ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3642026-05-16T04:29:06.948ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3652026-05-16T04:29:08.000ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3662026-05-16T04:29:08.004ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3672026-05-16T04:29:08.007ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3682026-05-16T04:29:09.139ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3692026-05-16T04:29:09.143ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3702026-05-16T04:29:09.146ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3712026-05-16T04:29:09.732ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3722026-05-16T04:29:09.737ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3732026-05-16T04:29:09.739ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3742026-05-16T04:29:11.174ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3752026-05-16T04:29:11.180ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3762026-05-16T04:29:11.183ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3772026-05-16T04:29:12.528ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3782026-05-16T04:29:12.533ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3792026-05-16T04:29:12.537ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3802026-05-16T04:29:13.654ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
3812026-05-16T04:29:13.690ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3822026-05-16T04:29:13.694ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3832026-05-16T04:29:13.698ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3842026-05-16T04:29:14.291ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3852026-05-16T04:29:14.296ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3862026-05-16T04:29:14.299ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3872026-05-16T04:29:14.952ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
3882026-05-16T04:29:15.546ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3892026-05-16T04:29:15.550ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3902026-05-16T04:29:15.554ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3912026-05-16T04:29:16.347ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3922026-05-16T04:29:16.348ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 102.711187833s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3932026-05-16T04:29:16.837ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3942026-05-16T04:29:16.842ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3952026-05-16T04:29:16.845ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3962026-05-16T04:29:18.082ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3972026-05-16T04:29:18.086ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
3982026-05-16T04:29:18.089ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3992026-05-16T04:29:19.067ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4002026-05-16T04:29:19.073ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4012026-05-16T04:29:19.076ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4022026-05-16T04:29:20.582ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4032026-05-16T04:29:20.586ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4042026-05-16T04:29:20.589ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4052026-05-16T04:29:22.024ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4062026-05-16T04:29:22.028ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4072026-05-16T04:29:22.031ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4082026-05-16T04:29:23.486ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4092026-05-16T04:29:23.490ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4102026-05-16T04:29:23.493ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4112026-05-16T04:29:23.683ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
4122026-05-16T04:29:24.403ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4132026-05-16T04:29:24.408ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4142026-05-16T04:29:24.412ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4152026-05-16T04:29:24.981ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
4162026-05-16T04:29:25.394ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4172026-05-16T04:29:25.399ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4182026-05-16T04:29:25.402ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4192026-05-16T04:29:26.099ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4202026-05-16T04:29:26.105ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4212026-05-16T04:29:26.108ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4222026-05-16T04:29:27.225ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4232026-05-16T04:29:27.231ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4242026-05-16T04:29:27.234ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4252026-05-16T04:29:27.891ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4262026-05-16T04:29:27.896ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4272026-05-16T04:29:27.899ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4282026-05-16T04:29:28.256ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("957682b6-3ca6-4e84-b11a-7dd8a8366649.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d1f6092c-34a6-4a6a-a2fe-73152fd5c29e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 779, tv_nsec: 44478668 } }) unit = producer-server
4292026-05-16T04:29:28.257ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
4302026-05-16T04:29:28.265ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
4312026-05-16T04:29:28.507ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4322026-05-16T04:29:28.511ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4332026-05-16T04:29:28.515ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4342026-05-16T04:29:29.866ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4352026-05-16T04:29:29.871ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4362026-05-16T04:29:29.874ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4372026-05-16T04:29:30.881ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4382026-05-16T04:29:30.885ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4392026-05-16T04:29:30.889ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4402026-05-16T04:29:31.521ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4412026-05-16T04:29:31.525ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4422026-05-16T04:29:31.528ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4432026-05-16T04:29:32.291ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4442026-05-16T04:29:32.295ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4452026-05-16T04:29:32.298ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4462026-05-16T04:29:33.494ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4472026-05-16T04:29:33.499ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4482026-05-16T04:29:33.503ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4492026-05-16T04:29:33.712ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
4502026-05-16T04:29:34.614ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4512026-05-16T04:29:34.619ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4522026-05-16T04:29:34.622ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4532026-05-16T04:29:35.010ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
4542026-05-16T04:29:35.303ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4552026-05-16T04:29:35.308ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4562026-05-16T04:29:35.312ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4572026-05-16T04:29:35.857ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4582026-05-16T04:29:35.862ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4592026-05-16T04:29:35.865ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4602026-05-16T04:29:37.379ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4612026-05-16T04:29:37.384ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4622026-05-16T04:29:37.387ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4632026-05-16T04:29:38.598ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4642026-05-16T04:29:38.603ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4652026-05-16T04:29:38.606ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4662026-05-16T04:29:39.610ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4672026-05-16T04:29:39.615ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4682026-05-16T04:29:39.619ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4692026-05-16T04:29:40.664ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4702026-05-16T04:29:40.668ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4712026-05-16T04:29:40.671ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4722026-05-16T04:29:41.351ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4732026-05-16T04:29:41.355ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4742026-05-16T04:29:41.359ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4752026-05-16T04:29:42.580ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4762026-05-16T04:29:42.585ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4772026-05-16T04:29:42.588ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4782026-05-16T04:29:43.307ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4792026-05-16T04:29:43.308ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 129.598442731s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
4802026-05-16T04:29:43.408ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4812026-05-16T04:29:43.414ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4822026-05-16T04:29:43.417ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4832026-05-16T04:29:43.741ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
4842026-05-16T04:29:44.885ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4852026-05-16T04:29:44.890ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4862026-05-16T04:29:44.894ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4872026-05-16T04:29:45.038ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
4882026-05-16T04:29:45.881ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4892026-05-16T04:29:45.885ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4902026-05-16T04:29:45.888ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4912026-05-16T04:29:47.084ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4922026-05-16T04:29:47.090ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4932026-05-16T04:29:47.092ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4942026-05-16T04:29:48.080ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4952026-05-16T04:29:48.084ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4962026-05-16T04:29:48.087ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4972026-05-16T04:29:48.988ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4982026-05-16T04:29:48.993ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
4992026-05-16T04:29:48.996ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5002026-05-16T04:29:50.214ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5012026-05-16T04:29:50.218ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5022026-05-16T04:29:50.222ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5032026-05-16T04:29:51.482ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5042026-05-16T04:29:51.486ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5052026-05-16T04:29:51.489ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5062026-05-16T04:29:52.432ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5072026-05-16T04:29:52.436ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5082026-05-16T04:29:52.440ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5092026-05-16T04:29:53.476ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5102026-05-16T04:29:53.480ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5112026-05-16T04:29:53.483ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5122026-05-16T04:29:53.770ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11490380268656351138 n_samples = 6 unit = oximeter
5132026-05-16T04:29:54.395ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("957682b6-3ca6-4e84-b11a-7dd8a8366649.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d1f6092c-34a6-4a6a-a2fe-73152fd5c29e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f9044a8b-5917-4c1f-9fa1-299a4ed626fd.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 805, tv_nsec: 112988373 } }) unit = producer-server
5142026-05-16T04:29:54.396ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
5152026-05-16T04:29:54.403ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
5162026-05-16T04:29:54.842ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5172026-05-16T04:29:54.847ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5182026-05-16T04:29:54.851ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5192026-05-16T04:29:55.066ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3444092742692835361 n_samples = 6 unit = oximeter
5202026-05-16T04:29:56.156ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5212026-05-16T04:29:56.162ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5222026-05-16T04:29:56.165ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5232026-05-16T04:29:56.893ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5242026-05-16T04:29:56.897ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5252026-05-16T04:29:56.901ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5262026-05-16T04:29:58.242ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5272026-05-16T04:29:58.247ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5282026-05-16T04:29:58.250ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5292026-05-16T04:29:59.226ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5302026-05-16T04:29:59.230ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5312026-05-16T04:29:59.233ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5322026-05-16T04:30:00.747ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5332026-05-16T04:30:00.752ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5342026-05-16T04:30:00.755ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5352026-05-16T04:30:01.623ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5362026-05-16T04:30:01.632ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::70c0:71ff:fec4:c764]
5372026-05-16T04:30:01.637ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]