Max level shown:
1 [ Oct 31 05:28:44 Enabled. ]
2 [ Oct 31 05:28:44 Rereading configuration. ]
3 [ Oct 31 05:28:45 Rereading configuration. ]
4 [ Oct 31 05:28:56 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Oct 31 05:28:56 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-10-31T05:28:56.639ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-10-31T05:28:56.641ZINFOtfportd: connected to dpd running softnpu-a133c73e
92026-10-31T05:28:56.641ZINFOtfportd: running in sync-only mode
102026-10-31T05:28:56.642ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-10-31T05:28:56.642ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-10-31T05:28:56.643ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( ffa852be-532b-4e85-9be1-1b389b0a94b3, ), sled_id: Some( 06130b34-0dc2-40b4-ba7f-f33f971a47f8, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
142026-10-31T05:28:56.643ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-10-31T05:28:56.649ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::cc06:95ff:fe3d:d57e->ce:06:95:3d:d5:7e interface = tfportrear0_0
162026-10-31T05:28:56.649ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::d40d:5eff:fe57:2b75->d6:0d:5e:57:2b:75 interface = tfportrear0_0
172026-10-31T05:28:56.650ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
182026-10-31T05:28:56.654ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
192026-10-31T05:28:56.659ZWARNtfportd: tfportqsfp0_0 has no addrs
20 [ Oct 31 05:28:57 Rereading configuration. ]
21 [ Oct 31 05:28:57 Executing refresh method (:kill). ]
22 [ Oct 31 05:28:57 Stopping because all processes in service exited. ]
23 [ Oct 31 05:28:57 Executing stop method (:kill). ]
24 [ Oct 31 05:28:57 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
25 [ Oct 31 05:28:57 Method "start" exited with status 0. ]
262026-10-31T05:28:57.279ZINFOtfportd: connecting to dpd host = [::1] port = 12224
272026-10-31T05:28:57.280ZINFOtfportd: connected to dpd running softnpu-a133c73e
282026-10-31T05:28:57.281ZINFOtfportd: running in sync-only mode
292026-10-31T05:28:57.281ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
302026-10-31T05:28:57.281ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
312026-10-31T05:28:57.281ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: ffa852be-532b-4e85-9be1-1b389b0a94b3, sled_id: 06130b34-0dc2-40b4-ba7f-f33f971a47f8, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 75de0b06-eb47-4bf7-b1ad-a15fd7cc84eb, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
322026-10-31T05:28:57.281ZDEBGtfportd: created producer ID producer_id = 8c65528c-b5ae-45c6-9033-4958dbdc57a1 unit = oximeter
332026-10-31T05:28:57.282ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( ffa852be-532b-4e85-9be1-1b389b0a94b3, ), sled_id: Some( 06130b34-0dc2-40b4-ba7f-f33f971a47f8, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
342026-10-31T05:28:57.298ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
352026-10-31T05:28:57.300ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::cc06:95ff:fe3d:d57e->ce:06:95:3d:d5:7e interface = tfportrear0_0
362026-10-31T05:28:57.300ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::d40d:5eff:fe57:2b75->d6:0d:5e:57:2b:75 interface = tfportrear0_0
372026-10-31T05:28:57.302ZDEBGtfportd: registered DTrace probes unit = producer-server
382026-10-31T05:28:57.302ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:61362 method = GET path = /{producer_id} unit = producer-server versions = all
392026-10-31T05:28:57.302ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:61362 unit = producer-server
402026-10-31T05:28:57.302ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:61362 unit = producer-server
412026-10-31T05:28:57.302ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:61362 unit = producer-server
422026-10-31T05:28:57.303ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
432026-10-31T05:28:57.303ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
442026-10-31T05:28:57.303ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
452026-10-31T05:28:57.303ZDEBGtfportd: starting producer registration task unit = producer-server
462026-10-31T05:28:57.303ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:61362 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 8c65528c-b5ae-45c6-9033-4958dbdc57a1 unit = producer-server
472026-10-31T05:28:57.303ZINFOtfportd: created oximeter producer server unit = oximeter
482026-10-31T05:28:57.303ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
492026-10-31T05:28:57.303ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
502026-10-31T05:28:57.304ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-10-31T05:28:40.145784130Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
512026-10-31T05:28:57.304ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 1666880144374177607 unit = oximeter
522026-10-31T05:28:57.304ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
532026-10-31T05:28:57.305ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("18fe29fb-d26c-4e0e-a4c9-7f8193db3854.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("daa8ad22-d0d3-480d-b059-f418b9d8209b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f3809b70-4699-4e19-b528-b9f28f4a874b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f3809b70-4699-4e19-b528-b9f28f4a874b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 3453, tv_nsec: 110127517 } }) unit = producer-server
542026-10-31T05:28:57.305ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
552026-10-31T05:28:57.320ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
562026-10-31T05:28:57.321ZWARNtfportd: tfportqsfp0_0 has no addrs
572026-10-31T05:28:58.117ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
582026-10-31T05:28:58.121ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
592026-10-31T05:28:58.124ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
602026-10-31T05:28:58.124ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
612026-10-31T05:28:58.125ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-10-31T05:28:40.145649538Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
622026-10-31T05:28:58.125ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 8209258497759838572 unit = oximeter
632026-10-31T05:28:58.125ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
642026-10-31T05:28:58.983ZINFOtfportd: added new ARP entry entry = 10.151.2.174->7a:b3:6e:ee:d1:05 interface = tfportqsfp0_0
652026-10-31T05:28:59.268ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
662026-10-31T05:28:59.273ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
672026-10-31T05:28:59.276ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
682026-10-31T05:29:00.063ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
692026-10-31T05:29:00.067ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
702026-10-31T05:29:00.069ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
712026-10-31T05:29:00.680ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
722026-10-31T05:29:00.685ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
732026-10-31T05:29:00.687ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
742026-10-31T05:29:02.010ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
752026-10-31T05:29:02.015ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
762026-10-31T05:29:02.017ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
772026-10-31T05:29:03.310ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
782026-10-31T05:29:03.315ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
792026-10-31T05:29:03.318ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
802026-10-31T05:29:03.965ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
812026-10-31T05:29:03.969ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
822026-10-31T05:29:03.972ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
832026-10-31T05:29:05.077ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
842026-10-31T05:29:05.081ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
852026-10-31T05:29:05.083ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
862026-10-31T05:29:06.262ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
872026-10-31T05:29:06.267ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
882026-10-31T05:29:06.270ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
892026-10-31T05:29:07.180ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902026-10-31T05:29:07.184ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
912026-10-31T05:29:07.187ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
922026-10-31T05:29:07.307ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1666880144374177607 n_samples = 6 unit = oximeter
932026-10-31T05:29:08.128ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8209258497759838572 n_samples = 6 unit = oximeter
942026-10-31T05:29:08.650ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
952026-10-31T05:29:08.656ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
962026-10-31T05:29:08.658ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
972026-10-31T05:29:09.794ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
982026-10-31T05:29:09.799ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
992026-10-31T05:29:09.802ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1002026-10-31T05:29:11.275ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1012026-10-31T05:29:11.279ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1022026-10-31T05:29:11.282ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1032026-10-31T05:29:12.370ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1042026-10-31T05:29:12.377ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1052026-10-31T05:29:12.381ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1062026-10-31T05:29:12.891ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1072026-10-31T05:29:12.892ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 15.019245569s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1082026-10-31T05:29:13.019ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("18fe29fb-d26c-4e0e-a4c9-7f8193db3854.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("daa8ad22-d0d3-480d-b059-f418b9d8209b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f3809b70-4699-4e19-b528-b9f28f4a874b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f3809b70-4699-4e19-b528-b9f28f4a874b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 3468, tv_nsec: 255696105 } }) unit = producer-server
1092026-10-31T05:29:13.020ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1102026-10-31T05:29:13.028ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1112026-10-31T05:29:13.320ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-10-31T05:29:13.327ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1132026-10-31T05:29:13.331ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1142026-10-31T05:29:14.096ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-10-31T05:29:14.101ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1162026-10-31T05:29:14.104ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1172026-10-31T05:29:14.726ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1182026-10-31T05:29:14.731ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1192026-10-31T05:29:14.734ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1202026-10-31T05:29:15.360ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1212026-10-31T05:29:15.365ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1222026-10-31T05:29:15.367ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1232026-10-31T05:29:15.883ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1242026-10-31T05:29:15.887ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1252026-10-31T05:29:15.889ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1262026-10-31T05:29:16.475ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1272026-10-31T05:29:16.487ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1282026-10-31T05:29:16.504ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1292026-10-31T05:29:17.111ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1302026-10-31T05:29:17.118ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1312026-10-31T05:29:17.123ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1322026-10-31T05:29:17.866ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1666880144374177607 n_samples = 6 unit = oximeter
1332026-10-31T05:29:18.283ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342026-10-31T05:29:18.288ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1352026-10-31T05:29:18.291ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1362026-10-31T05:29:18.684ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8209258497759838572 n_samples = 6 unit = oximeter
1372026-10-31T05:29:19.386ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1382026-10-31T05:29:19.391ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1392026-10-31T05:29:19.393ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1402026-10-31T05:29:20.541ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1412026-10-31T05:29:20.546ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1422026-10-31T05:29:20.548ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1432026-10-31T05:29:21.850ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1442026-10-31T05:29:21.854ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1452026-10-31T05:29:21.858ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1462026-10-31T05:29:23.210ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1472026-10-31T05:29:23.217ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1482026-10-31T05:29:23.221ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1492026-10-31T05:29:24.347ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1502026-10-31T05:29:24.351ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1512026-10-31T05:29:24.354ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1522026-10-31T05:29:25.444ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1532026-10-31T05:29:25.449ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1542026-10-31T05:29:25.452ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1552026-10-31T05:29:26.613ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1562026-10-31T05:29:26.619ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1572026-10-31T05:29:26.623ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1582026-10-31T05:29:27.580ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1592026-10-31T05:29:27.584ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1602026-10-31T05:29:27.586ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1612026-10-31T05:29:27.841ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1666880144374177607 n_samples = 6 unit = oximeter
1622026-10-31T05:29:28.029ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1632026-10-31T05:29:28.031ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 30.198119739s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1642026-10-31T05:29:28.412ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("18fe29fb-d26c-4e0e-a4c9-7f8193db3854.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("daa8ad22-d0d3-480d-b059-f418b9d8209b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f3809b70-4699-4e19-b528-b9f28f4a874b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f3809b70-4699-4e19-b528-b9f28f4a874b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 3483, tv_nsec: 690900225 } }) unit = producer-server
1652026-10-31T05:29:28.413ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1662026-10-31T05:29:28.419ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1672026-10-31T05:29:28.658ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8209258497759838572 n_samples = 6 unit = oximeter
1682026-10-31T05:29:29.065ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1692026-10-31T05:29:29.069ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1702026-10-31T05:29:29.072ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1712026-10-31T05:29:30.211ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-10-31T05:29:30.216ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1732026-10-31T05:29:30.219ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1742026-10-31T05:29:31.330ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1752026-10-31T05:29:31.335ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1762026-10-31T05:29:31.338ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1772026-10-31T05:29:32.465ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1782026-10-31T05:29:32.470ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1792026-10-31T05:29:32.472ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1802026-10-31T05:29:33.238ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1812026-10-31T05:29:33.244ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1822026-10-31T05:29:33.247ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1832026-10-31T05:29:33.992ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1842026-10-31T05:29:33.998ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1852026-10-31T05:29:34.001ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1862026-10-31T05:29:35.429ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1872026-10-31T05:29:35.436ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1882026-10-31T05:29:35.441ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1892026-10-31T05:29:36.845ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1902026-10-31T05:29:36.852ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1912026-10-31T05:29:36.857ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1922026-10-31T05:29:37.827ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1666880144374177607 n_samples = 6 unit = oximeter
1932026-10-31T05:29:38.146ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1942026-10-31T05:29:38.151ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1952026-10-31T05:29:38.154ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1962026-10-31T05:29:38.648ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8209258497759838572 n_samples = 6 unit = oximeter
1972026-10-31T05:29:39.253ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1982026-10-31T05:29:39.262ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
1992026-10-31T05:29:39.267ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2002026-10-31T05:29:41.345ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2012026-10-31T05:29:41.352ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2022026-10-31T05:29:41.356ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2032026-10-31T05:29:45.650ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2042026-10-31T05:29:45.655ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2052026-10-31T05:29:45.662ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2062026-10-31T05:29:45.758ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 47.943303411s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2072026-10-31T05:29:45.767ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2082026-10-31T05:29:47.077ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-10-31T05:29:47.088ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2102026-10-31T05:29:47.097ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2112026-10-31T05:29:47.118ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("18fe29fb-d26c-4e0e-a4c9-7f8193db3854.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("daa8ad22-d0d3-480d-b059-f418b9d8209b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f3809b70-4699-4e19-b528-b9f28f4a874b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f3809b70-4699-4e19-b528-b9f28f4a874b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 3502, tv_nsec: 410550662 } }) unit = producer-server
2122026-10-31T05:29:47.121ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2132026-10-31T05:29:47.174ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2142026-10-31T05:29:48.460ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1666880144374177607 n_samples = 6 unit = oximeter
2152026-10-31T05:29:48.488ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2162026-10-31T05:29:48.497ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2172026-10-31T05:29:48.501ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2182026-10-31T05:29:48.648ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8209258497759838572 n_samples = 6 unit = oximeter
2192026-10-31T05:29:49.948ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2202026-10-31T05:29:49.957ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2212026-10-31T05:29:49.962ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2222026-10-31T05:29:50.793ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2232026-10-31T05:29:50.802ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2242026-10-31T05:29:50.807ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2252026-10-31T05:29:51.698ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2262026-10-31T05:29:51.706ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2272026-10-31T05:29:51.710ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2282026-10-31T05:29:52.995ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2292026-10-31T05:29:53.005ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2302026-10-31T05:29:53.010ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2312026-10-31T05:29:53.846ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2322026-10-31T05:29:53.855ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2332026-10-31T05:29:53.859ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2342026-10-31T05:29:54.707ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2352026-10-31T05:29:54.716ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2362026-10-31T05:29:54.721ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2372026-10-31T05:29:55.300ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2382026-10-31T05:29:55.308ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2392026-10-31T05:29:55.313ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2402026-10-31T05:29:56.371ZINFOtfportd: removed stale NDP entry :fe80::cc06:95ff:fe3d:d57e->ce:06:95:3d:d5:7e
2412026-10-31T05:29:56.526ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2422026-10-31T05:29:56.532ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2432026-10-31T05:29:56.535ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2442026-10-31T05:29:57.493ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2452026-10-31T05:29:57.501ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2462026-10-31T05:29:57.505ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2472026-10-31T05:29:58.243ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::cc06:95ff:fe3d:d57e->ce:06:95:3d:d5:7e interface = tfportrear0_0
2482026-10-31T05:29:58.380ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2492026-10-31T05:29:58.388ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2502026-10-31T05:29:58.392ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2512026-10-31T05:29:58.465ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 1666880144374177607 n_samples = 6 unit = oximeter
2522026-10-31T05:29:58.652ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8209258497759838572 n_samples = 6 unit = oximeter
2532026-10-31T05:29:59.625ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2542026-10-31T05:29:59.633ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2552026-10-31T05:29:59.638ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2562026-10-31T05:30:00.460ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2572026-10-31T05:30:00.470ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2582026-10-31T05:30:00.475ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2592026-10-31T05:30:01.921ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2602026-10-31T05:30:01.931ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::d40d:5eff:fe57:2b75]
2612026-10-31T05:30:01.936ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2622026-10-31T05:30:02.175ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2632026-10-31T05:30:02.178ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 64.362639222s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server