Max level shown:
1 [ Jun 22 22:13:52 Enabled. ]
2 [ Jun 22 22:13:52 Rereading configuration. ]
3 [ Jun 22 22:13:53 Rereading configuration. ]
4 [ Jun 22 22:14:04 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jun 22 22:14:04 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-06-22T22:14:04.845ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-06-22T22:14:04.847ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
92026-06-22T22:14:04.847ZINFOtfportd: running in sync-only mode
102026-06-22T22:14:04.850ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-06-22T22:14:04.850ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-06-22T22:14:04.851ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 84f084d9-433f-4517-b608-f41abe585fed, ), sled_id: Some( c5e4938a-fadf-4b77-8fc1-0d727fea351e, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-06-22T22:14:04.851ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-06-22T22:14:04.856ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::700e:1ff:fe58:44df->72:0e:01:58:44:df interface = tfportrear0_0
162026-06-22T22:14:04.858ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-06-22T22:14:04.864ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
182026-06-22T22:14:04.869ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Jun 22 22:14:05 Rereading configuration. ]
20 [ Jun 22 22:14:05 Executing refresh method (:kill). ]
21 [ Jun 22 22:14:05 Stopping because all processes in service exited. ]
22 [ Jun 22 22:14:05 Executing stop method (:kill). ]
23 [ Jun 22 22:14:05 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Jun 22 22:14:05 Method "start" exited with status 0. ]
252026-06-22T22:14:05.485ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-06-22T22:14:05.486ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
272026-06-22T22:14:05.487ZINFOtfportd: running in sync-only mode
282026-06-22T22:14:05.487ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-06-22T22:14:05.489ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-06-22T22:14:05.489ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 84f084d9-433f-4517-b608-f41abe585fed, sled_id: c5e4938a-fadf-4b77-8fc1-0d727fea351e, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 8bd2512d-9baf-4e91-bedb-7c55e9cc3264, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-06-22T22:14:05.489ZDEBGtfportd: created producer ID producer_id = cac6467e-224f-40f1-8712-853d287e5897 unit = oximeter
322026-06-22T22:14:05.490ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 84f084d9-433f-4517-b608-f41abe585fed, ), sled_id: Some( c5e4938a-fadf-4b77-8fc1-0d727fea351e, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
332026-06-22T22:14:05.499ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-06-22T22:14:05.500ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::700e:1ff:fe58:44df->72:0e:01:58:44:df interface = tfportrear0_0
352026-06-22T22:14:05.501ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::10ec:45ff:fe24:12f5->12:ec:45:24:12:f5 interface = tfportrear0_0
362026-06-22T22:14:05.502ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-06-22T22:14:05.503ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:51751 method = GET path = /{producer_id} unit = producer-server versions = all
382026-06-22T22:14:05.503ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:51751 unit = producer-server
392026-06-22T22:14:05.503ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:51751 unit = producer-server
402026-06-22T22:14:05.503ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:51751 unit = producer-server
412026-06-22T22:14:05.503ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-06-22T22:14:05.503ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-06-22T22:14:05.504ZDEBGtfportd: starting producer registration task unit = producer-server
442026-06-22T22:14:05.504ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:51751 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = cac6467e-224f-40f1-8712-853d287e5897 unit = producer-server
452026-06-22T22:14:05.504ZINFOtfportd: created oximeter producer server unit = oximeter
462026-06-22T22:14:05.504ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-06-22T22:14:05.504ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
482026-06-22T22:14:05.505ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-06-22T22:14:05.505ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 1.123264ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
502026-06-22T22:14:05.506ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-06-22T22:13:48.583459806Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
512026-06-22T22:14:05.506ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 10893021871167817936 unit = oximeter
522026-06-22T22:14:05.506ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
532026-06-22T22:14:05.507ZWARNtfportd: tfportqsfp0_0 has no addrs
542026-06-22T22:14:05.632ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 128.358476ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
552026-06-22T22:14:06.303ZINFOtfportd: added new ARP entry entry = 10.151.1.174->5e:8e:6a:ef:c5:ee interface = tfportqsfp0_0
562026-06-22T22:14:06.332ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 828.135342ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
572026-06-22T22:14:06.707ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
582026-06-22T22:14:06.712ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
592026-06-22T22:14:06.715ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
602026-06-22T22:14:06.715ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
612026-06-22T22:14:06.716ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-06-22T22:13:48.583329793Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
622026-06-22T22:14:06.716ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 17889303807894568400 unit = oximeter
632026-06-22T22:14:06.716ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
642026-06-22T22:14:07.481ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8e6df471-5a81-4b5c-9644-274bf456a1e6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c827f6cc-2912-4ecf-ad68-944b0eed86ea.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dcbbdf34-0dce-4894-b25e-c53fc696f30a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dcbbdf34-0dce-4894-b25e-c53fc696f30a.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 708, tv_nsec: 184101500 } }) unit = producer-server
652026-06-22T22:14:07.481ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
662026-06-22T22:14:07.500ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
672026-06-22T22:14:07.801ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-06-22T22:14:07.806ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
692026-06-22T22:14:07.809ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
702026-06-22T22:14:08.417ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-06-22T22:14:08.423ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
722026-06-22T22:14:08.427ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
732026-06-22T22:14:08.979ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-06-22T22:14:08.984ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
752026-06-22T22:14:08.987ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
762026-06-22T22:14:09.539ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-06-22T22:14:09.545ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
782026-06-22T22:14:09.548ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
792026-06-22T22:14:11.056ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-06-22T22:14:11.060ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
812026-06-22T22:14:11.063ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
822026-06-22T22:14:11.941ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-06-22T22:14:11.945ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
842026-06-22T22:14:11.948ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
852026-06-22T22:14:12.553ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862026-06-22T22:14:12.558ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
872026-06-22T22:14:12.561ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
882026-06-22T22:14:13.721ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
892026-06-22T22:14:13.726ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
902026-06-22T22:14:13.729ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
912026-06-22T22:14:14.574ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
922026-06-22T22:14:14.578ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
932026-06-22T22:14:14.581ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
942026-06-22T22:14:15.509ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10893021871167817936 n_samples = 6 unit = oximeter
952026-06-22T22:14:15.734ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
962026-06-22T22:14:15.738ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
972026-06-22T22:14:15.742ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
982026-06-22T22:14:16.676ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
992026-06-22T22:14:16.681ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1002026-06-22T22:14:16.685ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1012026-06-22T22:14:16.718ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17889303807894568400 n_samples = 6 unit = oximeter
1022026-06-22T22:14:18.183ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-06-22T22:14:18.189ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1042026-06-22T22:14:18.192ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1052026-06-22T22:14:18.876ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-06-22T22:14:18.881ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1072026-06-22T22:14:18.885ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1082026-06-22T22:14:20.063ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-06-22T22:14:20.067ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1102026-06-22T22:14:20.071ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1112026-06-22T22:14:21.489ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-06-22T22:14:21.493ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1132026-06-22T22:14:21.496ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1142026-06-22T22:14:22.502ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1152026-06-22T22:14:22.503ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 16.998847665s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1162026-06-22T22:14:22.674ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1172026-06-22T22:14:22.678ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1182026-06-22T22:14:22.685ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1192026-06-22T22:14:23.297ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1202026-06-22T22:14:23.302ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1212026-06-22T22:14:23.305ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1222026-06-22T22:14:23.997ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1232026-06-22T22:14:24.000ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1242026-06-22T22:14:24.003ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1252026-06-22T22:14:25.383ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1262026-06-22T22:14:25.387ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1272026-06-22T22:14:25.390ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1282026-06-22T22:14:25.464ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8e6df471-5a81-4b5c-9644-274bf456a1e6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c827f6cc-2912-4ecf-ad68-944b0eed86ea.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dcbbdf34-0dce-4894-b25e-c53fc696f30a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dcbbdf34-0dce-4894-b25e-c53fc696f30a.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 725, tv_nsec: 952725039 } }) unit = producer-server
1292026-06-22T22:14:25.465ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1302026-06-22T22:14:25.472ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1312026-06-22T22:14:26.032ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10893021871167817936 n_samples = 6 unit = oximeter
1322026-06-22T22:14:26.301ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1332026-06-22T22:14:26.306ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1342026-06-22T22:14:26.308ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1352026-06-22T22:14:27.495ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17889303807894568400 n_samples = 6 unit = oximeter
1362026-06-22T22:14:28.004ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-06-22T22:14:28.009ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1382026-06-22T22:14:28.011ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1392026-06-22T22:14:28.591ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-06-22T22:14:28.597ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1412026-06-22T22:14:28.600ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1422026-06-22T22:14:29.279ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-06-22T22:14:29.282ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1442026-06-22T22:14:29.285ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1452026-06-22T22:14:30.530ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-06-22T22:14:30.534ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1472026-06-22T22:14:30.538ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1482026-06-22T22:14:31.310ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-06-22T22:14:31.315ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1502026-06-22T22:14:31.318ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1512026-06-22T22:14:32.737ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-06-22T22:14:32.741ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1532026-06-22T22:14:32.744ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1542026-06-22T22:14:33.689ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1552026-06-22T22:14:33.694ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1562026-06-22T22:14:33.697ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1572026-06-22T22:14:34.666ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1582026-06-22T22:14:34.676ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1592026-06-22T22:14:34.679ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1602026-06-22T22:14:35.222ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1612026-06-22T22:14:35.227ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1622026-06-22T22:14:35.230ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1632026-06-22T22:14:36.266ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10893021871167817936 n_samples = 6 unit = oximeter
1642026-06-22T22:14:36.385ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1652026-06-22T22:14:36.390ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1662026-06-22T22:14:36.393ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1672026-06-22T22:14:37.152ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1682026-06-22T22:14:37.156ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1692026-06-22T22:14:37.159ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1702026-06-22T22:14:37.472ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17889303807894568400 n_samples = 6 unit = oximeter
1712026-06-22T22:14:37.895ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-06-22T22:14:37.900ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1732026-06-22T22:14:37.903ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1742026-06-22T22:14:38.762ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1752026-06-22T22:14:38.767ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1762026-06-22T22:14:38.770ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1772026-06-22T22:14:40.224ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1782026-06-22T22:14:40.229ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1792026-06-22T22:14:40.232ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1802026-06-22T22:14:40.987ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1812026-06-22T22:14:40.991ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1822026-06-22T22:14:40.995ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1832026-06-22T22:14:40.998ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1842026-06-22T22:14:41.000ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 34.755002904s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1852026-06-22T22:14:42.378ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862026-06-22T22:14:42.383ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1872026-06-22T22:14:42.386ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1882026-06-22T22:14:43.513ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-06-22T22:14:43.517ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1902026-06-22T22:14:43.520ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1912026-06-22T22:14:44.123ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1922026-06-22T22:14:44.129ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1932026-06-22T22:14:44.133ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1942026-06-22T22:14:45.157ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1952026-06-22T22:14:45.162ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1962026-06-22T22:14:45.165ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1972026-06-22T22:14:46.075ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1982026-06-22T22:14:46.079ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
1992026-06-22T22:14:46.082ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2002026-06-22T22:14:46.241ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10893021871167817936 n_samples = 6 unit = oximeter
2012026-06-22T22:14:46.253ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8e6df471-5a81-4b5c-9644-274bf456a1e6.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c827f6cc-2912-4ecf-ad68-944b0eed86ea.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("dcbbdf34-0dce-4894-b25e-c53fc696f30a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dcbbdf34-0dce-4894-b25e-c53fc696f30a.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 746, tv_nsec: 230232094 } }) unit = producer-server
2022026-06-22T22:14:46.254ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2032026-06-22T22:14:46.262ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2042026-06-22T22:14:47.385ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2052026-06-22T22:14:47.391ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2062026-06-22T22:14:47.394ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2072026-06-22T22:14:47.447ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17889303807894568400 n_samples = 6 unit = oximeter
2082026-06-22T22:14:48.064ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-06-22T22:14:48.068ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2102026-06-22T22:14:48.071ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2112026-06-22T22:14:49.093ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-06-22T22:14:49.098ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2132026-06-22T22:14:49.102ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2142026-06-22T22:14:49.664ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2152026-06-22T22:14:49.669ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2162026-06-22T22:14:49.672ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2172026-06-22T22:14:50.718ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2182026-06-22T22:14:50.724ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2192026-06-22T22:14:50.727ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2202026-06-22T22:14:51.564ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2212026-06-22T22:14:51.569ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2222026-06-22T22:14:51.573ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2232026-06-22T22:14:52.953ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2242026-06-22T22:14:52.958ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2252026-06-22T22:14:52.961ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2262026-06-22T22:14:54.237ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2272026-06-22T22:14:54.243ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2282026-06-22T22:14:54.246ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2292026-06-22T22:14:55.180ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2302026-06-22T22:14:55.185ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2312026-06-22T22:14:55.188ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2322026-06-22T22:14:56.216ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10893021871167817936 n_samples = 6 unit = oximeter
2332026-06-22T22:14:56.610ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-06-22T22:14:56.614ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2352026-06-22T22:14:56.617ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2362026-06-22T22:14:57.329ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372026-06-22T22:14:57.334ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2382026-06-22T22:14:57.337ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2392026-06-22T22:14:57.422ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17889303807894568400 n_samples = 6 unit = oximeter
2402026-06-22T22:14:58.008ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2412026-06-22T22:14:58.013ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2422026-06-22T22:14:58.016ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2432026-06-22T22:14:58.912ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2442026-06-22T22:14:58.917ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2452026-06-22T22:14:58.921ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2462026-06-22T22:15:00.214ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2472026-06-22T22:15:00.219ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::700e:1ff:fe58:44df]
2482026-06-22T22:15:00.222ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2492026-06-22T22:15:01.223ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2502026-06-22T22:15:01.223ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 55.033849166s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server