Max level shown:
1 [ Jul 1 16:42:21 Enabled. ]
2 [ Jul 1 16:42:21 Rereading configuration. ]
3 [ Jul 1 16:42:21 Rereading configuration. ]
4 [ Jul 1 16:42:32 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jul 1 16:42:32 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-07-01T16:42:32.896ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-07-01T16:42:32.898ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
92026-07-01T16:42:32.898ZINFOtfportd: running in sync-only mode
102026-07-01T16:42:32.898ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-07-01T16:42:32.900ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-07-01T16:42:32.901ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 658e9e66-61b5-418f-9ef1-faf6b779f860, ), sled_id: Some( 96584ddc-23da-4e08-a5c0-7ef098d14ff0, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-07-01T16:42:32.901ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-07-01T16:42:32.907ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::fc88:9cff:fe76:803b->fe:88:9c:76:80:3b interface = tfportrear0_0
162026-07-01T16:42:32.908ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-07-01T16:42:32.913ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
182026-07-01T16:42:32.918ZWARNtfportd: tfportqsfp0_0 has no addrs
192026-07-01T16:42:33.495ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
202026-07-01T16:42:33.501ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
212026-07-01T16:42:33.505ZWARNtfportd: tfportqsfp0_0 has no addrs
22 [ Jul 1 16:42:33 Rereading configuration. ]
23 [ Jul 1 16:42:33 Executing refresh method (:kill). ]
24 [ Jul 1 16:42:33 Stopping because all processes in service exited. ]
25 [ Jul 1 16:42:33 Executing stop method (:kill). ]
26 [ Jul 1 16:42:33 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
27 [ Jul 1 16:42:33 Method "start" exited with status 0. ]
282026-07-01T16:42:33.583ZINFOtfportd: connecting to dpd host = [::1] port = 12224
292026-07-01T16:42:33.584ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
302026-07-01T16:42:33.584ZINFOtfportd: running in sync-only mode
312026-07-01T16:42:33.584ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
322026-07-01T16:42:33.586ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
332026-07-01T16:42:33.586ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 658e9e66-61b5-418f-9ef1-faf6b779f860, sled_id: 96584ddc-23da-4e08-a5c0-7ef098d14ff0, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 2536d261-2afb-4524-b215-8a9b0814d1c0, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
342026-07-01T16:42:33.586ZDEBGtfportd: created producer ID producer_id = d6e001db-6894-4aa3-8924-0e9d27bc6650 unit = oximeter
352026-07-01T16:42:33.587ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 658e9e66-61b5-418f-9ef1-faf6b779f860, ), sled_id: Some( 96584ddc-23da-4e08-a5c0-7ef098d14ff0, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
362026-07-01T16:42:33.594ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
372026-07-01T16:42:33.596ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::d844:bff:fe98:d656->da:44:0b:98:d6:56 interface = tfportrear0_0
382026-07-01T16:42:33.597ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::fc88:9cff:fe76:803b->fe:88:9c:76:80:3b interface = tfportrear0_0
392026-07-01T16:42:33.599ZDEBGtfportd: registered DTrace probes unit = producer-server
402026-07-01T16:42:33.600ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:38793 method = GET path = /{producer_id} unit = producer-server versions = all
412026-07-01T16:42:33.600ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:38793 unit = producer-server
422026-07-01T16:42:33.600ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:38793 unit = producer-server
432026-07-01T16:42:33.600ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:38793 unit = producer-server
442026-07-01T16:42:33.600ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
452026-07-01T16:42:33.600ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
462026-07-01T16:42:33.601ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
472026-07-01T16:42:33.601ZDEBGtfportd: starting producer registration task unit = producer-server
482026-07-01T16:42:33.601ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:38793 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = d6e001db-6894-4aa3-8924-0e9d27bc6650 unit = producer-server
492026-07-01T16:42:33.601ZINFOtfportd: created oximeter producer server unit = oximeter
502026-07-01T16:42:33.601ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
512026-07-01T16:42:33.601ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
522026-07-01T16:42:33.603ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1fe45cbc-be6d-445d-8fc8-1bf3531b9f98.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41530108-1c87-4ce8-a255-6bf24455f3c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 771, tv_nsec: 44164548 } }) unit = producer-server
532026-07-01T16:42:33.603ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-07-01T16:42:16.761453625Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
542026-07-01T16:42:33.603ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 14693410002482495354 unit = oximeter
552026-07-01T16:42:33.603ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
562026-07-01T16:42:33.603ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
572026-07-01T16:42:33.621ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
582026-07-01T16:42:33.621ZWARNtfportd: tfportqsfp0_0 has no addrs
592026-07-01T16:42:34.852ZINFOtfportd: added new ARP entry entry = 10.151.1.174->f6:08:b7:82:ab:8b interface = tfportqsfp0_0
602026-07-01T16:42:34.986ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
612026-07-01T16:42:34.992ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
622026-07-01T16:42:34.995ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
632026-07-01T16:42:34.996ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
642026-07-01T16:42:34.997ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-07-01T16:42:16.761321863Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
652026-07-01T16:42:34.997ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 12384129854332682257 unit = oximeter
662026-07-01T16:42:34.997ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
672026-07-01T16:42:36.120ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-07-01T16:42:36.125ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
692026-07-01T16:42:36.128ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
702026-07-01T16:42:36.913ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-07-01T16:42:36.918ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
722026-07-01T16:42:36.921ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
732026-07-01T16:42:37.903ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-07-01T16:42:37.909ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
752026-07-01T16:42:37.912ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
762026-07-01T16:42:38.422ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-07-01T16:42:38.428ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
782026-07-01T16:42:38.432ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
792026-07-01T16:42:39.275ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-07-01T16:42:39.279ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
812026-07-01T16:42:39.288ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
822026-07-01T16:42:40.119ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-07-01T16:42:40.124ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
842026-07-01T16:42:40.128ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
852026-07-01T16:42:40.803ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862026-07-01T16:42:40.809ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
872026-07-01T16:42:40.813ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
882026-07-01T16:42:41.825ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
892026-07-01T16:42:41.830ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
902026-07-01T16:42:41.833ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
912026-07-01T16:42:42.735ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
922026-07-01T16:42:42.740ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
932026-07-01T16:42:42.743ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
942026-07-01T16:42:43.439ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
952026-07-01T16:42:43.443ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
962026-07-01T16:42:43.446ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
972026-07-01T16:42:43.606ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
982026-07-01T16:42:44.522ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
992026-07-01T16:42:44.526ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1002026-07-01T16:42:44.529ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1012026-07-01T16:42:44.999ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
1022026-07-01T16:42:45.825ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-07-01T16:42:45.830ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1042026-07-01T16:42:45.833ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1052026-07-01T16:42:46.825ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-07-01T16:42:46.830ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1072026-07-01T16:42:46.834ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1082026-07-01T16:42:47.371ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-07-01T16:42:47.376ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1102026-07-01T16:42:47.380ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1112026-07-01T16:42:49.248ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1122026-07-01T16:42:49.249ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 15.021774069s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1132026-07-01T16:42:49.329ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1142026-07-01T16:42:49.334ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1152026-07-01T16:42:49.337ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1162026-07-01T16:42:49.424ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1fe45cbc-be6d-445d-8fc8-1bf3531b9f98.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41530108-1c87-4ce8-a255-6bf24455f3c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 786, tv_nsec: 239659556 } }) unit = producer-server
1172026-07-01T16:42:49.425ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1182026-07-01T16:42:49.433ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1192026-07-01T16:42:50.818ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1202026-07-01T16:42:50.827ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1212026-07-01T16:42:50.830ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1222026-07-01T16:42:51.979ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1232026-07-01T16:42:51.984ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1242026-07-01T16:42:51.987ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1252026-07-01T16:42:52.937ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1262026-07-01T16:42:52.941ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1272026-07-01T16:42:52.945ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1282026-07-01T16:42:54.223ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
1292026-07-01T16:42:54.423ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1302026-07-01T16:42:54.427ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1312026-07-01T16:42:54.430ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1322026-07-01T16:42:55.558ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1332026-07-01T16:42:55.563ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1342026-07-01T16:42:55.566ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1352026-07-01T16:42:55.612ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
1362026-07-01T16:42:56.248ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-07-01T16:42:56.253ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1382026-07-01T16:42:56.256ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1392026-07-01T16:42:56.924ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-07-01T16:42:56.928ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1412026-07-01T16:42:56.931ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1422026-07-01T16:42:58.286ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-07-01T16:42:58.291ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1442026-07-01T16:42:58.294ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1452026-07-01T16:42:59.374ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-07-01T16:42:59.379ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1472026-07-01T16:42:59.382ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1482026-07-01T16:43:00.796ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-07-01T16:43:00.800ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1502026-07-01T16:43:00.804ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1512026-07-01T16:43:01.584ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-07-01T16:43:01.588ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1532026-07-01T16:43:01.591ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1542026-07-01T16:43:02.127ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1552026-07-01T16:43:02.132ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1562026-07-01T16:43:02.135ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1572026-07-01T16:43:03.139ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1582026-07-01T16:43:03.144ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1592026-07-01T16:43:03.148ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1602026-07-01T16:43:03.946ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1612026-07-01T16:43:03.965ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1622026-07-01T16:43:03.974ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1632026-07-01T16:43:04.201ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
1642026-07-01T16:43:04.398ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1652026-07-01T16:43:04.398ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 30.20853962s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1662026-07-01T16:43:05.073ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1fe45cbc-be6d-445d-8fc8-1bf3531b9f98.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41530108-1c87-4ce8-a255-6bf24455f3c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 801, tv_nsec: 925959326 } }) unit = producer-server
1672026-07-01T16:43:05.077ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1682026-07-01T16:43:05.092ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1692026-07-01T16:43:05.244ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1702026-07-01T16:43:05.252ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1712026-07-01T16:43:05.258ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1722026-07-01T16:43:05.588ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
1732026-07-01T16:43:06.235ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1742026-07-01T16:43:06.244ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1752026-07-01T16:43:06.249ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1762026-07-01T16:43:07.735ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1772026-07-01T16:43:07.743ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1782026-07-01T16:43:07.747ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1792026-07-01T16:43:08.314ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1802026-07-01T16:43:08.320ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1812026-07-01T16:43:08.323ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1822026-07-01T16:43:09.627ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1832026-07-01T16:43:09.632ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1842026-07-01T16:43:09.636ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1852026-07-01T16:43:10.898ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862026-07-01T16:43:10.902ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1872026-07-01T16:43:10.905ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1882026-07-01T16:43:12.211ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-07-01T16:43:12.218ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1902026-07-01T16:43:12.222ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1912026-07-01T16:43:13.542ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1922026-07-01T16:43:13.546ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1932026-07-01T16:43:13.549ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1942026-07-01T16:43:14.176ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
1952026-07-01T16:43:14.480ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1962026-07-01T16:43:14.485ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
1972026-07-01T16:43:14.488ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1982026-07-01T16:43:15.134ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1992026-07-01T16:43:15.139ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2002026-07-01T16:43:15.142ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2012026-07-01T16:43:15.563ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
2022026-07-01T16:43:16.028ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032026-07-01T16:43:16.033ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2042026-07-01T16:43:16.036ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2052026-07-01T16:43:16.593ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2062026-07-01T16:43:16.598ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2072026-07-01T16:43:16.602ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2082026-07-01T16:43:17.915ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-07-01T16:43:17.920ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2102026-07-01T16:43:17.923ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2112026-07-01T16:43:19.022ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-07-01T16:43:19.028ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2132026-07-01T16:43:19.031ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2142026-07-01T16:43:20.085ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2152026-07-01T16:43:20.214ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 46.064220166s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2162026-07-01T16:43:20.564ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2172026-07-01T16:43:20.575ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2182026-07-01T16:43:20.582ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2192026-07-01T16:43:21.220ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1fe45cbc-be6d-445d-8fc8-1bf3531b9f98.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41530108-1c87-4ce8-a255-6bf24455f3c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 818, tv_nsec: 114719495 } }) unit = producer-server
2202026-07-01T16:43:21.306ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2212026-07-01T16:43:21.306ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2222026-07-01T16:43:21.574ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2232026-07-01T16:43:21.581ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2242026-07-01T16:43:21.585ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2252026-07-01T16:43:22.872ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2262026-07-01T16:43:22.879ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2272026-07-01T16:43:22.882ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2282026-07-01T16:43:23.663ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2292026-07-01T16:43:23.667ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2302026-07-01T16:43:23.670ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2312026-07-01T16:43:24.165ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
2322026-07-01T16:43:24.374ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2332026-07-01T16:43:24.379ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2342026-07-01T16:43:24.382ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2352026-07-01T16:43:25.432ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2362026-07-01T16:43:25.437ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2372026-07-01T16:43:25.440ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2382026-07-01T16:43:25.556ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
2392026-07-01T16:43:26.882ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402026-07-01T16:43:26.887ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2412026-07-01T16:43:26.890ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2422026-07-01T16:43:27.821ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2432026-07-01T16:43:27.825ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2442026-07-01T16:43:27.828ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2452026-07-01T16:43:28.856ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462026-07-01T16:43:28.861ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2472026-07-01T16:43:28.864ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2482026-07-01T16:43:30.071ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2492026-07-01T16:43:30.076ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2502026-07-01T16:43:30.079ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2512026-07-01T16:43:30.871ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2522026-07-01T16:43:30.876ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2532026-07-01T16:43:30.880ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2542026-07-01T16:43:32.262ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2552026-07-01T16:43:32.267ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2562026-07-01T16:43:32.270ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2572026-07-01T16:43:33.661ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2582026-07-01T16:43:33.666ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2592026-07-01T16:43:33.669ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2602026-07-01T16:43:34.169ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
2612026-07-01T16:43:34.671ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2622026-07-01T16:43:34.680ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2632026-07-01T16:43:34.682ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2642026-07-01T16:43:35.388ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2652026-07-01T16:43:35.393ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2662026-07-01T16:43:35.395ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2672026-07-01T16:43:35.559ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
2682026-07-01T16:43:36.105ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692026-07-01T16:43:36.111ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2702026-07-01T16:43:36.114ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2712026-07-01T16:43:36.236ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2722026-07-01T16:43:36.237ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 62.085278617s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2732026-07-01T16:43:36.737ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2742026-07-01T16:43:36.742ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2752026-07-01T16:43:36.745ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2762026-07-01T16:43:38.073ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2772026-07-01T16:43:38.078ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2782026-07-01T16:43:38.081ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2792026-07-01T16:43:38.265ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1fe45cbc-be6d-445d-8fc8-1bf3531b9f98.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41530108-1c87-4ce8-a255-6bf24455f3c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 835, tv_nsec: 155052658 } }) unit = producer-server
2802026-07-01T16:43:38.265ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2812026-07-01T16:43:38.272ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2822026-07-01T16:43:39.251ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2832026-07-01T16:43:39.256ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2842026-07-01T16:43:39.259ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2852026-07-01T16:43:40.151ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2862026-07-01T16:43:40.154ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2872026-07-01T16:43:40.157ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2882026-07-01T16:43:40.685ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2892026-07-01T16:43:40.690ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2902026-07-01T16:43:40.693ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2912026-07-01T16:43:41.325ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2922026-07-01T16:43:41.334ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2932026-07-01T16:43:41.337ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2942026-07-01T16:43:42.694ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2952026-07-01T16:43:42.698ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2962026-07-01T16:43:42.702ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2972026-07-01T16:43:43.423ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2982026-07-01T16:43:43.428ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
2992026-07-01T16:43:43.431ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3002026-07-01T16:43:43.981ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3012026-07-01T16:43:43.991ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3022026-07-01T16:43:43.994ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3032026-07-01T16:43:44.172ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
3042026-07-01T16:43:44.587ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3052026-07-01T16:43:44.592ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3062026-07-01T16:43:44.595ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3072026-07-01T16:43:45.560ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
3082026-07-01T16:43:45.595ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3092026-07-01T16:43:45.601ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3102026-07-01T16:43:45.604ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3112026-07-01T16:43:46.813ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3122026-07-01T16:43:46.819ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3132026-07-01T16:43:46.822ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3142026-07-01T16:43:48.174ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3152026-07-01T16:43:48.180ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3162026-07-01T16:43:48.183ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3172026-07-01T16:43:48.882ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3182026-07-01T16:43:48.887ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3192026-07-01T16:43:48.890ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3202026-07-01T16:43:49.613ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3212026-07-01T16:43:49.618ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3222026-07-01T16:43:49.622ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3232026-07-01T16:43:50.434ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3242026-07-01T16:43:50.439ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3252026-07-01T16:43:50.442ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3262026-07-01T16:43:51.768ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3272026-07-01T16:43:51.774ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3282026-07-01T16:43:51.776ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3292026-07-01T16:43:52.301ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3302026-07-01T16:43:52.306ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3312026-07-01T16:43:52.308ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3322026-07-01T16:43:52.944ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3332026-07-01T16:43:52.949ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3342026-07-01T16:43:52.953ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3352026-07-01T16:43:53.272ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3362026-07-01T16:43:53.273ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 79.121847626s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3372026-07-01T16:43:53.725ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382026-07-01T16:43:53.731ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3392026-07-01T16:43:53.734ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3402026-07-01T16:43:54.173ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
3412026-07-01T16:43:54.671ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3422026-07-01T16:43:54.677ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3432026-07-01T16:43:54.680ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3442026-07-01T16:43:55.562ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
3452026-07-01T16:43:56.133ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3462026-07-01T16:43:56.139ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3472026-07-01T16:43:56.143ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3482026-07-01T16:43:56.271ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1fe45cbc-be6d-445d-8fc8-1bf3531b9f98.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41530108-1c87-4ce8-a255-6bf24455f3c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 853, tv_nsec: 162112168 } }) unit = producer-server
3492026-07-01T16:43:56.272ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
3502026-07-01T16:43:56.279ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3512026-07-01T16:43:56.726ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3522026-07-01T16:43:56.732ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3532026-07-01T16:43:56.735ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3542026-07-01T16:43:57.516ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3552026-07-01T16:43:57.522ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3562026-07-01T16:43:57.525ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3572026-07-01T16:43:58.255ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3582026-07-01T16:43:58.261ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3592026-07-01T16:43:58.263ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3602026-07-01T16:43:58.883ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3612026-07-01T16:43:58.889ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3622026-07-01T16:43:58.895ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3632026-07-01T16:44:00.114ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3642026-07-01T16:44:00.120ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3652026-07-01T16:44:00.123ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3662026-07-01T16:44:00.835ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3672026-07-01T16:44:00.841ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3682026-07-01T16:44:00.844ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3692026-07-01T16:44:01.890ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3702026-07-01T16:44:01.896ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3712026-07-01T16:44:01.899ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3722026-07-01T16:44:02.575ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3732026-07-01T16:44:02.580ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3742026-07-01T16:44:02.584ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3752026-07-01T16:44:03.749ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3762026-07-01T16:44:03.753ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3772026-07-01T16:44:03.757ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3782026-07-01T16:44:04.176ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
3792026-07-01T16:44:05.198ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3802026-07-01T16:44:05.203ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3812026-07-01T16:44:05.206ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3822026-07-01T16:44:05.565ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
3832026-07-01T16:44:06.165ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3842026-07-01T16:44:06.170ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3852026-07-01T16:44:06.173ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3862026-07-01T16:44:07.667ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3872026-07-01T16:44:07.679ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3882026-07-01T16:44:07.684ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3892026-07-01T16:44:08.654ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3902026-07-01T16:44:08.664ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3912026-07-01T16:44:08.669ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3922026-07-01T16:44:16.142ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3932026-07-01T16:44:16.161ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
3942026-07-01T16:44:16.161ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3952026-07-01T16:44:16.164ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
3962026-07-01T16:44:16.194ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 102.041448831s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3972026-07-01T16:44:16.240ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
3982026-07-01T16:44:16.288ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3992026-07-01T16:44:17.646ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4002026-07-01T16:44:17.661ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4012026-07-01T16:44:17.801ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4022026-07-01T16:44:18.931ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4032026-07-01T16:44:18.939ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4042026-07-01T16:44:18.945ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4052026-07-01T16:44:19.518ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4062026-07-01T16:44:19.527ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4072026-07-01T16:44:19.532ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4082026-07-01T16:44:20.294ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4092026-07-01T16:44:20.307ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4102026-07-01T16:44:20.316ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4112026-07-01T16:44:21.583ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4122026-07-01T16:44:21.593ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4132026-07-01T16:44:21.601ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4142026-07-01T16:44:22.421ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4152026-07-01T16:44:22.429ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4162026-07-01T16:44:22.434ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4172026-07-01T16:44:23.600ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4182026-07-01T16:44:23.606ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4192026-07-01T16:44:23.609ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4202026-07-01T16:44:24.338ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4212026-07-01T16:44:24.344ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4222026-07-01T16:44:24.348ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4232026-07-01T16:44:25.788ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4242026-07-01T16:44:25.795ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4252026-07-01T16:44:25.800ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4262026-07-01T16:44:26.165ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
4272026-07-01T16:44:26.167ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
4282026-07-01T16:44:26.558ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4292026-07-01T16:44:26.566ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4302026-07-01T16:44:26.571ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4312026-07-01T16:44:27.336ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4322026-07-01T16:44:27.346ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4332026-07-01T16:44:27.350ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4342026-07-01T16:44:27.716ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1fe45cbc-be6d-445d-8fc8-1bf3531b9f98.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41530108-1c87-4ce8-a255-6bf24455f3c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 884, tv_nsec: 603350674 } }) unit = producer-server
4352026-07-01T16:44:27.717ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
4362026-07-01T16:44:27.937ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
4372026-07-01T16:44:28.776ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4382026-07-01T16:44:28.784ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4392026-07-01T16:44:28.788ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4402026-07-01T16:44:29.956ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4412026-07-01T16:44:29.962ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4422026-07-01T16:44:29.966ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4432026-07-01T16:44:31.331ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4442026-07-01T16:44:31.337ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4452026-07-01T16:44:31.341ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4462026-07-01T16:44:32.853ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4472026-07-01T16:44:32.859ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4482026-07-01T16:44:32.863ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4492026-07-01T16:44:34.090ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4502026-07-01T16:44:34.098ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4512026-07-01T16:44:34.103ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4522026-07-01T16:44:34.997ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4532026-07-01T16:44:35.007ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4542026-07-01T16:44:35.011ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4552026-07-01T16:44:36.169ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
4562026-07-01T16:44:36.169ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
4572026-07-01T16:44:36.498ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4582026-07-01T16:44:36.504ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4592026-07-01T16:44:36.507ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4602026-07-01T16:44:37.845ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4612026-07-01T16:44:37.851ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4622026-07-01T16:44:37.855ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4632026-07-01T16:44:38.785ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4642026-07-01T16:44:38.791ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4652026-07-01T16:44:38.796ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4662026-07-01T16:44:40.188ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4672026-07-01T16:44:40.194ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4682026-07-01T16:44:40.197ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4692026-07-01T16:44:41.044ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4702026-07-01T16:44:41.050ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4712026-07-01T16:44:41.054ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4722026-07-01T16:44:41.949ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4732026-07-01T16:44:41.955ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4742026-07-01T16:44:41.959ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4752026-07-01T16:44:42.938ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4762026-07-01T16:44:42.940ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 128.787409491s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
4772026-07-01T16:44:43.220ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4782026-07-01T16:44:43.226ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4792026-07-01T16:44:43.229ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4802026-07-01T16:44:44.565ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4812026-07-01T16:44:44.572ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4822026-07-01T16:44:44.576ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4832026-07-01T16:44:45.185ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4842026-07-01T16:44:45.192ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4852026-07-01T16:44:45.195ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4862026-07-01T16:44:45.784ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4872026-07-01T16:44:45.790ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4882026-07-01T16:44:45.794ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4892026-07-01T16:44:46.171ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
4902026-07-01T16:44:46.172ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
4912026-07-01T16:44:46.421ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4922026-07-01T16:44:46.426ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4932026-07-01T16:44:46.429ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4942026-07-01T16:44:47.560ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4952026-07-01T16:44:47.566ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4962026-07-01T16:44:47.569ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4972026-07-01T16:44:48.176ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4982026-07-01T16:44:48.182ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
4992026-07-01T16:44:48.186ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5002026-07-01T16:44:48.756ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5012026-07-01T16:44:48.763ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5022026-07-01T16:44:48.767ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5032026-07-01T16:44:49.621ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5042026-07-01T16:44:49.627ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5052026-07-01T16:44:49.632ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5062026-07-01T16:44:51.048ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5072026-07-01T16:44:51.055ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5082026-07-01T16:44:51.058ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5092026-07-01T16:44:51.934ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5102026-07-01T16:44:51.945ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5112026-07-01T16:44:51.950ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5122026-07-01T16:44:52.489ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5132026-07-01T16:44:52.499ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5142026-07-01T16:44:52.504ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5152026-07-01T16:44:53.747ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5162026-07-01T16:44:53.754ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5172026-07-01T16:44:53.758ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5182026-07-01T16:44:55.075ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5192026-07-01T16:44:55.083ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5202026-07-01T16:44:55.087ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5212026-07-01T16:44:55.967ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5222026-07-01T16:44:55.975ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5232026-07-01T16:44:55.979ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5242026-07-01T16:44:56.038ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1fe45cbc-be6d-445d-8fc8-1bf3531b9f98.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("41530108-1c87-4ce8-a255-6bf24455f3c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3858899-534e-4430-b29f-329b8238fbf7.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 912, tv_nsec: 928284970 } }) unit = producer-server
5252026-07-01T16:44:56.039ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
5262026-07-01T16:44:56.046ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
5272026-07-01T16:44:56.136ZDEBGtfportd (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "ec069c40-a665-484c-83fe-ef27895fea5d", "content-length": "41", "date": "Wed, 01 Jul 2026 16:44:56 GMT"} }) unit = producer-server
5282026-07-01T16:44:56.138ZDEBGtfportd (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s unit = producer-server wait_period = 150s
5292026-07-01T16:44:56.174ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14693410002482495354 n_samples = 6 unit = oximeter
5302026-07-01T16:44:56.175ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12384129854332682257 n_samples = 6 unit = oximeter
5312026-07-01T16:44:57.027ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5322026-07-01T16:44:57.034ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5332026-07-01T16:44:57.038ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5342026-07-01T16:44:57.577ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5352026-07-01T16:44:57.583ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5362026-07-01T16:44:57.586ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5372026-07-01T16:44:59.034ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5382026-07-01T16:44:59.041ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5392026-07-01T16:44:59.045ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5402026-07-01T16:45:00.492ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5412026-07-01T16:45:00.500ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5422026-07-01T16:45:00.506ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5432026-07-01T16:45:01.083ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5442026-07-01T16:45:01.094ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5452026-07-01T16:45:01.100ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5462026-07-01T16:45:01.729ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5472026-07-01T16:45:01.738ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5482026-07-01T16:45:01.743ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5492026-07-01T16:45:02.425ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5502026-07-01T16:45:02.436ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::fc88:9cff:fe76:803b]
5512026-07-01T16:45:02.442ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]