Max level shown:
1 [ Aug 20 04:43:04 Enabled. ]
2 [ Aug 20 04:43:04 Rereading configuration. ]
3 [ Aug 20 04:43:04 Rereading configuration. ]
4 [ Aug 20 04:43:15 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Aug 20 04:43:15 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-08-20T04:43:15.881ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-08-20T04:43:15.882ZINFOtfportd: connected to dpd running softnpu-5cb3f22d
92026-08-20T04:43:15.883ZINFOtfportd: running in sync-only mode
102026-08-20T04:43:15.884ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-08-20T04:43:15.885ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-08-20T04:43:15.887ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 0e27c49e-ff2f-41aa-86c6-ebdbf0b35203, ), sled_id: Some( 8aafcef4-7c39-4eb1-a847-e16be5ae5d17, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
142026-08-20T04:43:15.887ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-08-20T04:43:15.891ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::9c98:5dff:fe81:c925->9e:98:5d:81:c9:25 interface = tfportrear0_0
162026-08-20T04:43:15.893ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-08-20T04:43:15.898ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
182026-08-20T04:43:15.905ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Aug 20 04:43:16 Rereading configuration. ]
20 [ Aug 20 04:43:16 Executing refresh method (:kill). ]
21 [ Aug 20 04:43:16 Stopping because all processes in service exited. ]
22 [ Aug 20 04:43:16 Executing stop method (:kill). ]
23 [ Aug 20 04:43:16 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Aug 20 04:43:16 Method "start" exited with status 0. ]
252026-08-20T04:43:16.542ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-08-20T04:43:16.543ZINFOtfportd: connected to dpd running softnpu-5cb3f22d
272026-08-20T04:43:16.543ZINFOtfportd: running in sync-only mode
282026-08-20T04:43:16.544ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-08-20T04:43:16.545ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-08-20T04:43:16.546ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 0e27c49e-ff2f-41aa-86c6-ebdbf0b35203, sled_id: 8aafcef4-7c39-4eb1-a847-e16be5ae5d17, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 6cfdc304-be67-4d5b-a818-69349c7abcf8, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-08-20T04:43:16.546ZDEBGtfportd: created producer ID producer_id = e875c96e-3d71-4240-8699-63848967b3ec unit = oximeter
322026-08-20T04:43:16.547ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 0e27c49e-ff2f-41aa-86c6-ebdbf0b35203, ), sled_id: Some( 8aafcef4-7c39-4eb1-a847-e16be5ae5d17, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
332026-08-20T04:43:16.554ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-08-20T04:43:16.556ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::e0ca:20ff:fe94:505b->e2:ca:20:94:50:5b interface = tfportrear0_0
352026-08-20T04:43:16.556ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::9c98:5dff:fe81:c925->9e:98:5d:81:c9:25 interface = tfportrear0_0
362026-08-20T04:43:16.558ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-08-20T04:43:16.559ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:60084 method = GET path = /{producer_id} unit = producer-server versions = all
382026-08-20T04:43:16.559ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:60084 unit = producer-server
392026-08-20T04:43:16.559ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:60084 unit = producer-server
402026-08-20T04:43:16.559ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:60084 unit = producer-server
412026-08-20T04:43:16.559ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-08-20T04:43:16.559ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-08-20T04:43:16.559ZDEBGtfportd: starting producer registration task unit = producer-server
442026-08-20T04:43:16.560ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:60084 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = e875c96e-3d71-4240-8699-63848967b3ec unit = producer-server
452026-08-20T04:43:16.560ZINFOtfportd: created oximeter producer server unit = oximeter
462026-08-20T04:43:16.560ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-08-20T04:43:16.560ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
482026-08-20T04:43:16.560ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-08-20T04:43:16.561ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 1.249861ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
502026-08-20T04:43:16.562ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-08-20T04:42:59.677437471Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
512026-08-20T04:43:16.562ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 8299592397180796555 unit = oximeter
522026-08-20T04:43:16.562ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
532026-08-20T04:43:16.563ZWARNtfportd: tfportqsfp0_0 has no addrs
542026-08-20T04:43:16.764ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 204.012295ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
552026-08-20T04:43:17.058ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("615b3c85-be91-4522-a949-73b0ffd17c34.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("92e7f004-a738-4a50-9b33-735fd7d721db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 722, tv_nsec: 530710806 } }) unit = producer-server
562026-08-20T04:43:17.059ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
572026-08-20T04:43:17.075ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
582026-08-20T04:43:17.092ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
592026-08-20T04:43:17.099ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
602026-08-20T04:43:17.103ZWARNtfportd: tfportqsfp0_0 has no addrs
612026-08-20T04:43:17.944ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-08-20T04:43:17.949ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
632026-08-20T04:43:17.952ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
642026-08-20T04:43:17.953ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
652026-08-20T04:43:17.954ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-08-20T04:42:59.677301650Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
662026-08-20T04:43:17.954ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 1784381639250839747 unit = oximeter
672026-08-20T04:43:17.954ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
682026-08-20T04:43:17.959ZINFOtfportd: added new ARP entry entry = 10.151.2.174->c6:73:b6:4f:e0:e2 interface = tfportqsfp0_0
692026-08-20T04:43:19.231ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
702026-08-20T04:43:19.237ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
712026-08-20T04:43:19.240ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
722026-08-20T04:43:20.026ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
732026-08-20T04:43:20.032ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
742026-08-20T04:43:20.035ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
752026-08-20T04:43:20.852ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
762026-08-20T04:43:20.857ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
772026-08-20T04:43:20.860ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
782026-08-20T04:43:22.080ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
792026-08-20T04:43:22.085ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
802026-08-20T04:43:22.088ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
812026-08-20T04:43:22.976ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
822026-08-20T04:43:22.980ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
832026-08-20T04:43:22.983ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
842026-08-20T04:43:24.246ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
852026-08-20T04:43:24.251ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
862026-08-20T04:43:24.254ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
872026-08-20T04:43:25.238ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
882026-08-20T04:43:25.243ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
892026-08-20T04:43:25.246ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
902026-08-20T04:43:25.913ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
912026-08-20T04:43:25.918ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
922026-08-20T04:43:25.921ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
932026-08-20T04:43:26.436ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942026-08-20T04:43:26.441ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
952026-08-20T04:43:26.444ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
962026-08-20T04:43:26.565ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8299592397180796555 n_samples = 6 unit = oximeter
972026-08-20T04:43:27.008ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
982026-08-20T04:43:27.013ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
992026-08-20T04:43:27.016ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1002026-08-20T04:43:27.731ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1012026-08-20T04:43:27.736ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1022026-08-20T04:43:27.740ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1032026-08-20T04:43:27.957ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1784381639250839747 n_samples = 6 unit = oximeter
1042026-08-20T04:43:29.111ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1052026-08-20T04:43:29.115ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1062026-08-20T04:43:29.118ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1072026-08-20T04:43:29.681ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1082026-08-20T04:43:29.686ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1092026-08-20T04:43:29.689ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1102026-08-20T04:43:30.751ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1112026-08-20T04:43:30.757ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1122026-08-20T04:43:30.760ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1132026-08-20T04:43:31.966ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1142026-08-20T04:43:31.971ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1152026-08-20T04:43:31.974ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1162026-08-20T04:43:32.079ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1172026-08-20T04:43:32.081ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 15.51820548s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1182026-08-20T04:43:32.808ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1192026-08-20T04:43:32.813ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1202026-08-20T04:43:32.817ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1212026-08-20T04:43:32.853ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("615b3c85-be91-4522-a949-73b0ffd17c34.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("92e7f004-a738-4a50-9b33-735fd7d721db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 738, tv_nsec: 320856613 } }) unit = producer-server
1222026-08-20T04:43:32.854ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
1232026-08-20T04:43:32.862ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1242026-08-20T04:43:33.947ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1252026-08-20T04:43:33.952ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1262026-08-20T04:43:33.955ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1272026-08-20T04:43:35.414ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282026-08-20T04:43:35.419ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1292026-08-20T04:43:35.422ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1302026-08-20T04:43:36.137ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312026-08-20T04:43:36.142ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1322026-08-20T04:43:36.146ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1332026-08-20T04:43:36.584ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8299592397180796555 n_samples = 6 unit = oximeter
1342026-08-20T04:43:36.769ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1352026-08-20T04:43:36.774ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1362026-08-20T04:43:36.777ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1372026-08-20T04:43:37.341ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1382026-08-20T04:43:37.345ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1392026-08-20T04:43:37.348ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1402026-08-20T04:43:37.978ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1784381639250839747 n_samples = 6 unit = oximeter
1412026-08-20T04:43:38.034ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1422026-08-20T04:43:38.038ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1432026-08-20T04:43:38.041ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1442026-08-20T04:43:39.273ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1452026-08-20T04:43:39.282ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1462026-08-20T04:43:39.287ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1472026-08-20T04:43:39.977ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1482026-08-20T04:43:39.983ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1492026-08-20T04:43:39.986ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1502026-08-20T04:43:41.267ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1512026-08-20T04:43:41.273ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1522026-08-20T04:43:41.276ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1532026-08-20T04:43:42.632ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1542026-08-20T04:43:42.658ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1552026-08-20T04:43:42.720ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1562026-08-20T04:43:44.075ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1572026-08-20T04:43:44.085ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1582026-08-20T04:43:44.090ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1592026-08-20T04:43:45.688ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1602026-08-20T04:43:45.698ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1612026-08-20T04:43:45.702ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1622026-08-20T04:43:46.615ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8299592397180796555 n_samples = 6 unit = oximeter
1632026-08-20T04:43:46.969ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1642026-08-20T04:43:46.981ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1652026-08-20T04:43:46.985ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1662026-08-20T04:43:47.572ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1672026-08-20T04:43:47.576ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1682026-08-20T04:43:47.580ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1692026-08-20T04:43:47.905ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1702026-08-20T04:43:47.906ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 31.299449596s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1712026-08-20T04:43:48.006ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1784381639250839747 n_samples = 6 unit = oximeter
1722026-08-20T04:43:48.603ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1732026-08-20T04:43:48.608ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1742026-08-20T04:43:48.611ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1752026-08-20T04:43:49.324ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1762026-08-20T04:43:49.330ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1772026-08-20T04:43:49.333ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1782026-08-20T04:43:49.681ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("615b3c85-be91-4522-a949-73b0ffd17c34.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("92e7f004-a738-4a50-9b33-735fd7d721db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 755, tv_nsec: 101757147 } }) unit = producer-server
1792026-08-20T04:43:49.682ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
1802026-08-20T04:43:49.688ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1812026-08-20T04:43:50.547ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1822026-08-20T04:43:50.553ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1832026-08-20T04:43:50.556ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1842026-08-20T04:43:51.625ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1852026-08-20T04:43:51.630ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1862026-08-20T04:43:51.633ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1872026-08-20T04:43:52.739ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1882026-08-20T04:43:52.744ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1892026-08-20T04:43:52.747ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1902026-08-20T04:43:53.685ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1912026-08-20T04:43:53.691ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1922026-08-20T04:43:53.694ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1932026-08-20T04:43:54.399ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1942026-08-20T04:43:54.404ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1952026-08-20T04:43:54.407ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1962026-08-20T04:43:55.564ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1972026-08-20T04:43:55.569ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
1982026-08-20T04:43:55.572ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1992026-08-20T04:43:56.597ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-08-20T04:43:56.602ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2012026-08-20T04:43:56.604ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2022026-08-20T04:43:56.644ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8299592397180796555 n_samples = 6 unit = oximeter
2032026-08-20T04:43:57.607ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2042026-08-20T04:43:57.613ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2052026-08-20T04:43:57.616ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2062026-08-20T04:43:58.032ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1784381639250839747 n_samples = 6 unit = oximeter
2072026-08-20T04:43:58.866ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2082026-08-20T04:43:58.871ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2092026-08-20T04:43:58.875ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2102026-08-20T04:44:00.110ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2112026-08-20T04:44:00.116ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2122026-08-20T04:44:00.123ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2132026-08-20T04:44:00.872ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2142026-08-20T04:44:00.877ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2152026-08-20T04:44:00.880ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2162026-08-20T04:44:01.960ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2172026-08-20T04:44:01.966ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2182026-08-20T04:44:01.969ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2192026-08-20T04:44:02.551ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2202026-08-20T04:44:02.557ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2212026-08-20T04:44:02.562ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2222026-08-20T04:44:03.189ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2232026-08-20T04:44:03.195ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2242026-08-20T04:44:03.199ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2252026-08-20T04:44:04.599ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2262026-08-20T04:44:04.606ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2272026-08-20T04:44:04.610ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2282026-08-20T04:44:04.707ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2292026-08-20T04:44:04.708ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 48.07972453s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2302026-08-20T04:44:06.079ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2312026-08-20T04:44:06.086ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2322026-08-20T04:44:06.091ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2332026-08-20T04:44:06.646ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8299592397180796555 n_samples = 6 unit = oximeter
2342026-08-20T04:44:06.839ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("615b3c85-be91-4522-a949-73b0ffd17c34.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("92e7f004-a738-4a50-9b33-735fd7d721db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 772, tv_nsec: 241088762 } }) unit = producer-server
2352026-08-20T04:44:06.839ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
2362026-08-20T04:44:06.850ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
2372026-08-20T04:44:07.164ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2382026-08-20T04:44:07.170ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2392026-08-20T04:44:07.173ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2402026-08-20T04:44:08.032ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1784381639250839747 n_samples = 6 unit = oximeter
2412026-08-20T04:44:08.583ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2422026-08-20T04:44:08.588ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2432026-08-20T04:44:08.591ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2442026-08-20T04:44:10.018ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2452026-08-20T04:44:10.024ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2462026-08-20T04:44:10.027ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2472026-08-20T04:44:10.956ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2482026-08-20T04:44:10.962ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2492026-08-20T04:44:10.965ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2502026-08-20T04:44:12.058ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2512026-08-20T04:44:12.064ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2522026-08-20T04:44:12.068ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2532026-08-20T04:44:12.849ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2542026-08-20T04:44:12.871ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2552026-08-20T04:44:12.875ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2562026-08-20T04:44:14.795ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2572026-08-20T04:44:14.811ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2582026-08-20T04:44:16.929ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2592026-08-20T04:44:17.063ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8299592397180796555 n_samples = 6 unit = oximeter
2602026-08-20T04:44:18.035ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1784381639250839747 n_samples = 6 unit = oximeter
2612026-08-20T04:44:18.501ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2622026-08-20T04:44:18.589ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2632026-08-20T04:44:18.609ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2642026-08-20T04:44:20.243ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2652026-08-20T04:44:20.252ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2662026-08-20T04:44:20.257ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2672026-08-20T04:44:26.071ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2682026-08-20T04:44:26.086ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2692026-08-20T04:44:26.109ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2702026-08-20T04:44:26.119ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 69.490218606s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2712026-08-20T04:44:26.226ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2722026-08-20T04:44:27.088ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8299592397180796555 n_samples = 6 unit = oximeter
2732026-08-20T04:44:27.238ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2742026-08-20T04:44:27.246ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2752026-08-20T04:44:27.270ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2762026-08-20T04:44:28.040ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1784381639250839747 n_samples = 6 unit = oximeter
2772026-08-20T04:44:28.179ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2782026-08-20T04:44:28.187ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2792026-08-20T04:44:28.192ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2802026-08-20T04:44:29.252ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2812026-08-20T04:44:29.263ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2822026-08-20T04:44:29.270ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2832026-08-20T04:44:30.744ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2842026-08-20T04:44:30.754ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2852026-08-20T04:44:30.761ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2862026-08-20T04:44:31.773ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2872026-08-20T04:44:31.799ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2882026-08-20T04:44:31.817ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2892026-08-20T04:44:32.820ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2902026-08-20T04:44:32.830ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2912026-08-20T04:44:32.836ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2922026-08-20T04:44:33.717ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2932026-08-20T04:44:33.726ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
2942026-08-20T04:44:33.731ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2952026-08-20T04:44:34.437ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("615b3c85-be91-4522-a949-73b0ffd17c34.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("92e7f004-a738-4a50-9b33-735fd7d721db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 799, tv_nsec: 836248166 } }) unit = producer-server
2962026-08-20T04:44:34.439ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
2972026-08-20T04:44:34.505ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
2982026-08-20T04:44:34.955ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2992026-08-20T04:44:34.963ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3002026-08-20T04:44:34.967ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3012026-08-20T04:44:35.822ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3022026-08-20T04:44:35.830ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3032026-08-20T04:44:35.834ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3042026-08-20T04:44:36.612ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3052026-08-20T04:44:36.619ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3062026-08-20T04:44:36.624ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3072026-08-20T04:44:37.092ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8299592397180796555 n_samples = 6 unit = oximeter
3082026-08-20T04:44:37.368ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3092026-08-20T04:44:37.379ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3102026-08-20T04:44:37.384ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3112026-08-20T04:44:38.043ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1784381639250839747 n_samples = 6 unit = oximeter
3122026-08-20T04:44:38.120ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3132026-08-20T04:44:38.130ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3142026-08-20T04:44:38.135ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3152026-08-20T04:44:39.453ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3162026-08-20T04:44:39.462ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3172026-08-20T04:44:39.468ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3182026-08-20T04:44:40.870ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3192026-08-20T04:44:40.880ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3202026-08-20T04:44:40.884ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3212026-08-20T04:44:42.358ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3222026-08-20T04:44:42.366ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3232026-08-20T04:44:42.370ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3242026-08-20T04:44:43.766ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3252026-08-20T04:44:43.772ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3262026-08-20T04:44:43.776ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3272026-08-20T04:44:44.701ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3282026-08-20T04:44:44.708ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3292026-08-20T04:44:44.711ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3302026-08-20T04:44:45.525ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3312026-08-20T04:44:45.530ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3322026-08-20T04:44:45.533ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3332026-08-20T04:44:46.418ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3342026-08-20T04:44:46.423ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3352026-08-20T04:44:46.426ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3362026-08-20T04:44:47.095ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8299592397180796555 n_samples = 6 unit = oximeter
3372026-08-20T04:44:47.102ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382026-08-20T04:44:47.112ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3392026-08-20T04:44:47.115ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3402026-08-20T04:44:48.046ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1784381639250839747 n_samples = 6 unit = oximeter
3412026-08-20T04:44:48.576ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3422026-08-20T04:44:48.582ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3432026-08-20T04:44:48.585ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3442026-08-20T04:44:49.483ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3452026-08-20T04:44:49.489ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3462026-08-20T04:44:49.492ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3472026-08-20T04:44:49.506ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3482026-08-20T04:44:49.507ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 92.878596828s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3492026-08-20T04:44:50.119ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3502026-08-20T04:44:50.125ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3512026-08-20T04:44:50.129ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3522026-08-20T04:44:51.357ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3532026-08-20T04:44:51.363ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3542026-08-20T04:44:51.366ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3552026-08-20T04:44:52.594ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3562026-08-20T04:44:52.602ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3572026-08-20T04:44:52.607ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3582026-08-20T04:44:53.197ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3592026-08-20T04:44:53.206ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3602026-08-20T04:44:53.212ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3612026-08-20T04:44:53.932ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3622026-08-20T04:44:53.939ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3632026-08-20T04:44:53.943ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3642026-08-20T04:44:54.791ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3652026-08-20T04:44:54.797ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3662026-08-20T04:44:54.800ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3672026-08-20T04:44:55.604ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3682026-08-20T04:44:55.609ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3692026-08-20T04:44:55.612ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3702026-08-20T04:44:56.752ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3712026-08-20T04:44:56.758ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3722026-08-20T04:44:56.762ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3732026-08-20T04:44:57.098ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8299592397180796555 n_samples = 6 unit = oximeter
3742026-08-20T04:44:57.603ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3752026-08-20T04:44:57.609ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3762026-08-20T04:44:57.612ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3772026-08-20T04:44:57.983ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("615b3c85-be91-4522-a949-73b0ffd17c34.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("92e7f004-a738-4a50-9b33-735fd7d721db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("b38209a6-f947-4434-8b11-5ccaae993961.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 823, tv_nsec: 386562651 } }) unit = producer-server
3782026-08-20T04:44:57.983ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
3792026-08-20T04:44:57.990ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
3802026-08-20T04:44:58.050ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1784381639250839747 n_samples = 6 unit = oximeter
3812026-08-20T04:44:58.051ZDEBGtfportd (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "89d4cf19-ac7b-421d-90ae-884ede4b1f4b", "content-length": "41", "date": "Thu, 20 Aug 2026 04:44:58 GMT"} }) unit = producer-server
3822026-08-20T04:44:58.053ZDEBGtfportd (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s unit = producer-server wait_period = 150s
3832026-08-20T04:44:58.439ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3842026-08-20T04:44:58.444ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3852026-08-20T04:44:58.447ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3862026-08-20T04:44:59.393ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3872026-08-20T04:44:59.398ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3882026-08-20T04:44:59.400ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3892026-08-20T04:45:00.354ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3902026-08-20T04:45:00.361ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9c98:5dff:fe81:c925]
3912026-08-20T04:45:00.365ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]