Max level shown:
1 [ Aug 15 07:28:49 Enabled. ]
2 [ Aug 15 07:28:49 Rereading configuration. ]
3 [ Aug 15 07:28:50 Rereading configuration. ]
4 [ Aug 15 07:29:01 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Aug 15 07:29:01 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-08-15T07:29:01.158ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-08-15T07:29:01.160ZINFOtfportd: connected to dpd running softnpu-5cb3f22d
92026-08-15T07:29:01.161ZINFOtfportd: running in sync-only mode
102026-08-15T07:29:01.162ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-08-15T07:29:01.163ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-08-15T07:29:01.165ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 0874b1c5-8b60-4231-88a9-f85d15cf2cdf, ), sled_id: Some( 6f82c21f-e982-497d-b0fc-6c2cabea1dff, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-08-15T07:29:01.165ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-08-15T07:29:01.169ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::9011:15ff:fe77:fb6->92:11:15:77:0f:b6 interface = tfportrear0_0
162026-08-15T07:29:01.171ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-08-15T07:29:01.176ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
182026-08-15T07:29:01.182ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Aug 15 07:29:01 Rereading configuration. ]
20 [ Aug 15 07:29:01 Executing refresh method (:kill). ]
21 [ Aug 15 07:29:01 Stopping because all processes in service exited. ]
22 [ Aug 15 07:29:01 Executing stop method (:kill). ]
23 [ Aug 15 07:29:01 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Aug 15 07:29:01 Method "start" exited with status 0. ]
252026-08-15T07:29:01.822ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-08-15T07:29:01.823ZINFOtfportd: connected to dpd running softnpu-5cb3f22d
272026-08-15T07:29:01.823ZINFOtfportd: running in sync-only mode
282026-08-15T07:29:01.823ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-08-15T07:29:01.825ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-08-15T07:29:01.826ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 0874b1c5-8b60-4231-88a9-f85d15cf2cdf, sled_id: 6f82c21f-e982-497d-b0fc-6c2cabea1dff, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 1272e73a-1953-477d-bf82-53e5903f7448, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-08-15T07:29:01.826ZDEBGtfportd: created producer ID producer_id = 758494e0-7bb1-4c31-87d7-c7942e1a4568 unit = oximeter
322026-08-15T07:29:01.827ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 0874b1c5-8b60-4231-88a9-f85d15cf2cdf, ), sled_id: Some( 6f82c21f-e982-497d-b0fc-6c2cabea1dff, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
332026-08-15T07:29:01.835ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-08-15T07:29:01.836ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::9011:15ff:fe77:fb6->92:11:15:77:0f:b6 interface = tfportrear0_0
352026-08-15T07:29:01.836ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::54b2:41ff:fed4:640c->56:b2:41:d4:64:0c interface = tfportrear0_0
362026-08-15T07:29:01.838ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-08-15T07:29:01.839ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:36204 method = GET path = /{producer_id} unit = producer-server versions = all
382026-08-15T07:29:01.839ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:36204 unit = producer-server
392026-08-15T07:29:01.839ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:36204 unit = producer-server
402026-08-15T07:29:01.839ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:36204 unit = producer-server
412026-08-15T07:29:01.839ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-08-15T07:29:01.839ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-08-15T07:29:01.840ZDEBGtfportd: starting producer registration task unit = producer-server
442026-08-15T07:29:01.840ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:36204 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 758494e0-7bb1-4c31-87d7-c7942e1a4568 unit = producer-server
452026-08-15T07:29:01.840ZINFOtfportd: created oximeter producer server unit = oximeter
462026-08-15T07:29:01.840ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-08-15T07:29:01.840ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
482026-08-15T07:29:01.840ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-08-15T07:29:01.841ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 1.276629ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
502026-08-15T07:29:01.842ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-08-15T07:28:44.866571973Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
512026-08-15T07:29:01.842ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 2556501112265819910 unit = oximeter
522026-08-15T07:29:01.842ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
532026-08-15T07:29:01.843ZWARNtfportd: tfportqsfp0_0 has no addrs
542026-08-15T07:29:02.059ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 219.535935ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
552026-08-15T07:29:02.454ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 614.314917ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
562026-08-15T07:29:02.854ZINFOtfportd: added new ARP entry entry = 10.151.1.174->2e:c7:9a:9b:65:ab interface = tfportqsfp0_0
572026-08-15T07:29:03.246ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
582026-08-15T07:29:03.251ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
592026-08-15T07:29:03.254ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
602026-08-15T07:29:03.254ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
612026-08-15T07:29:03.255ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-08-15T07:28:44.866440700Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
622026-08-15T07:29:03.255ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 12580035396232926397 unit = oximeter
632026-08-15T07:29:03.255ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
642026-08-15T07:29:03.441ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 1.600839508s error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
652026-08-15T07:29:03.973ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
662026-08-15T07:29:03.977ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
672026-08-15T07:29:03.981ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
682026-08-15T07:29:05.146ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
692026-08-15T07:29:05.152ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
702026-08-15T07:29:05.155ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
712026-08-15T07:29:06.028ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
722026-08-15T07:29:06.033ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
732026-08-15T07:29:06.036ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
742026-08-15T07:29:06.420ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9fd222e9-936c-4b45-b166-9c8538a19ccd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ad80b418-70c2-4518-9143-4a4b7931d8d0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ae91ec61-769e-4c5e-8d2b-17fb66117971.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ae91ec61-769e-4c5e-8d2b-17fb66117971.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 18244, tv_nsec: 688079024 } }) unit = producer-server
752026-08-15T07:29:06.420ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
762026-08-15T07:29:06.436ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
772026-08-15T07:29:06.947ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
782026-08-15T07:29:06.952ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
792026-08-15T07:29:06.955ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
802026-08-15T07:29:07.505ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
812026-08-15T07:29:07.509ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
822026-08-15T07:29:07.512ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
832026-08-15T07:29:08.966ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
842026-08-15T07:29:08.971ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
852026-08-15T07:29:08.974ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
862026-08-15T07:29:09.501ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
872026-08-15T07:29:09.506ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
882026-08-15T07:29:09.509ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
892026-08-15T07:29:10.229ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902026-08-15T07:29:10.235ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
912026-08-15T07:29:10.237ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
922026-08-15T07:29:11.678ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
932026-08-15T07:29:11.683ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
942026-08-15T07:29:11.686ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
952026-08-15T07:29:11.845ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 2556501112265819910 n_samples = 6 unit = oximeter
962026-08-15T07:29:12.865ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-08-15T07:29:12.870ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
982026-08-15T07:29:12.873ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
992026-08-15T07:29:13.256ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12580035396232926397 n_samples = 6 unit = oximeter
1002026-08-15T07:29:13.533ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1012026-08-15T07:29:13.538ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1022026-08-15T07:29:13.542ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1032026-08-15T07:29:14.988ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1042026-08-15T07:29:14.993ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1052026-08-15T07:29:14.996ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1062026-08-15T07:29:16.484ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1072026-08-15T07:29:16.490ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1082026-08-15T07:29:16.493ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1092026-08-15T07:29:17.435ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1102026-08-15T07:29:17.439ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1112026-08-15T07:29:17.442ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1122026-08-15T07:29:18.031ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1132026-08-15T07:29:18.036ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1142026-08-15T07:29:18.039ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1152026-08-15T07:29:19.451ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1162026-08-15T07:29:19.456ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1172026-08-15T07:29:19.460ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1182026-08-15T07:29:20.842ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1192026-08-15T07:29:20.847ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1202026-08-15T07:29:20.850ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1212026-08-15T07:29:21.085ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1222026-08-15T07:29:21.086ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 19.599094953s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1232026-08-15T07:29:21.226ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1242026-08-15T07:29:21.231ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1252026-08-15T07:29:21.238ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1262026-08-15T07:29:21.494ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 2556501112265819910 n_samples = 6 unit = oximeter
1272026-08-15T07:29:22.035ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282026-08-15T07:29:22.040ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1292026-08-15T07:29:22.043ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1302026-08-15T07:29:22.908ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12580035396232926397 n_samples = 6 unit = oximeter
1312026-08-15T07:29:23.129ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1322026-08-15T07:29:23.134ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1332026-08-15T07:29:23.137ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1342026-08-15T07:29:23.398ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9fd222e9-936c-4b45-b166-9c8538a19ccd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ad80b418-70c2-4518-9143-4a4b7931d8d0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ae91ec61-769e-4c5e-8d2b-17fb66117971.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ae91ec61-769e-4c5e-8d2b-17fb66117971.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 18262, tv_nsec: 15542937 } }) unit = producer-server
1352026-08-15T07:29:23.398ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1362026-08-15T07:29:23.406ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1372026-08-15T07:29:24.247ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1382026-08-15T07:29:24.252ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1392026-08-15T07:29:24.255ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1402026-08-15T07:29:25.296ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1412026-08-15T07:29:25.301ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1422026-08-15T07:29:25.304ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1432026-08-15T07:29:26.057ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1442026-08-15T07:29:26.062ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1452026-08-15T07:29:26.065ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1462026-08-15T07:29:27.465ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1472026-08-15T07:29:27.470ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1482026-08-15T07:29:27.473ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1492026-08-15T07:29:28.079ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1502026-08-15T07:29:28.100ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1512026-08-15T07:29:28.113ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1522026-08-15T07:29:28.802ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1532026-08-15T07:29:28.810ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1542026-08-15T07:29:28.815ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1552026-08-15T07:29:30.003ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1562026-08-15T07:29:30.009ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1572026-08-15T07:29:30.012ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1582026-08-15T07:29:31.372ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1592026-08-15T07:29:31.378ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1602026-08-15T07:29:31.381ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1612026-08-15T07:29:31.501ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 2556501112265819910 n_samples = 6 unit = oximeter
1622026-08-15T07:29:32.237ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1632026-08-15T07:29:33.225ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12580035396232926397 n_samples = 6 unit = oximeter
1642026-08-15T07:29:33.226ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1652026-08-15T07:29:33.243ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1662026-08-15T07:29:33.934ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1672026-08-15T07:29:33.940ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1682026-08-15T07:29:33.949ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1692026-08-15T07:29:35.306ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1702026-08-15T07:29:35.311ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1712026-08-15T07:29:35.314ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1722026-08-15T07:29:36.721ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1732026-08-15T07:29:36.726ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1742026-08-15T07:29:36.729ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1752026-08-15T07:29:38.138ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1762026-08-15T07:29:38.143ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1772026-08-15T07:29:38.146ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1782026-08-15T07:29:38.408ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1792026-08-15T07:29:38.410ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 36.918727363s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1802026-08-15T07:29:39.246ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1812026-08-15T07:29:39.250ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1822026-08-15T07:29:39.253ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1832026-08-15T07:29:39.899ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1842026-08-15T07:29:39.905ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1852026-08-15T07:29:39.908ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1862026-08-15T07:29:40.603ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1872026-08-15T07:29:40.609ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1882026-08-15T07:29:40.612ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1892026-08-15T07:29:41.184ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1902026-08-15T07:29:41.191ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1912026-08-15T07:29:41.195ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1922026-08-15T07:29:41.504ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 2556501112265819910 n_samples = 6 unit = oximeter
1932026-08-15T07:29:42.059ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1942026-08-15T07:29:42.064ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1952026-08-15T07:29:42.067ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1962026-08-15T07:29:42.618ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1972026-08-15T07:29:42.623ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
1982026-08-15T07:29:42.626ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1992026-08-15T07:29:43.226ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12580035396232926397 n_samples = 6 unit = oximeter
2002026-08-15T07:29:43.598ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9fd222e9-936c-4b45-b166-9c8538a19ccd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ad80b418-70c2-4518-9143-4a4b7931d8d0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ae91ec61-769e-4c5e-8d2b-17fb66117971.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("ae91ec61-769e-4c5e-8d2b-17fb66117971.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 18282, tv_nsec: 215474199 } }) unit = producer-server
2012026-08-15T07:29:43.598ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
2022026-08-15T07:29:43.606ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
2032026-08-15T07:29:43.805ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2042026-08-15T07:29:43.810ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2052026-08-15T07:29:43.814ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2062026-08-15T07:29:44.775ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2072026-08-15T07:29:44.780ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2082026-08-15T07:29:44.783ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2092026-08-15T07:29:45.697ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2102026-08-15T07:29:45.703ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2112026-08-15T07:29:45.707ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2122026-08-15T07:29:46.750ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2132026-08-15T07:29:46.754ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2142026-08-15T07:29:46.758ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2152026-08-15T07:29:47.731ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2162026-08-15T07:29:47.737ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2172026-08-15T07:29:47.741ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2182026-08-15T07:29:48.637ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2192026-08-15T07:29:48.642ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2202026-08-15T07:29:48.646ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2212026-08-15T07:29:50.454ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2222026-08-15T07:29:50.464ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2232026-08-15T07:29:50.470ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2242026-08-15T07:29:51.508ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 2556501112265819910 n_samples = 6 unit = oximeter
2252026-08-15T07:29:51.843ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2262026-08-15T07:29:51.851ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2272026-08-15T07:29:51.855ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2282026-08-15T07:29:53.228ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 12580035396232926397 n_samples = 6 unit = oximeter
2292026-08-15T07:29:53.309ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2302026-08-15T07:29:53.314ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2312026-08-15T07:29:53.317ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2322026-08-15T07:29:53.900ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2332026-08-15T07:29:53.905ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2342026-08-15T07:29:53.909ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2352026-08-15T07:29:54.987ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2362026-08-15T07:29:54.992ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2372026-08-15T07:29:54.995ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2382026-08-15T07:29:56.076ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2392026-08-15T07:29:56.081ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2402026-08-15T07:29:56.084ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2412026-08-15T07:29:57.456ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2422026-08-15T07:29:57.460ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2432026-08-15T07:29:57.462ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2442026-08-15T07:29:58.379ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2452026-08-15T07:29:58.384ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2462026-08-15T07:29:58.387ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2472026-08-15T07:29:58.607ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2482026-08-15T07:29:58.608ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 57.116972858s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2492026-08-15T07:29:58.969ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2502026-08-15T07:29:58.973ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2512026-08-15T07:29:58.976ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2522026-08-15T07:29:59.953ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2532026-08-15T07:29:59.958ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::9011:15ff:fe77:fb6]
2542026-08-15T07:29:59.961ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]