Max level shown:
1 [ Jul 22 20:43:48 Enabled. ]
2 [ Jul 22 20:43:48 Rereading configuration. ]
3 [ Jul 22 20:43:48 Rereading configuration. ]
4 [ Jul 22 20:44:00 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jul 22 20:44:00 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-07-22T20:44:00.046ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-07-22T20:44:00.048ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
92026-07-22T20:44:00.049ZINFOtfportd: running in sync-only mode
102026-07-22T20:44:00.051ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-07-22T20:44:00.051ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-07-22T20:44:00.052ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 4d2618f7-a654-441e-b117-49ad2b7cc819, ), sled_id: Some( fe94a6c0-d682-4570-be03-b85617774e5b, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-07-22T20:44:00.052ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-07-22T20:44:00.058ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::ec87:55ff:fe31:3f13->ee:87:55:31:3f:13 interface = tfportrear0_0
162026-07-22T20:44:00.059ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-07-22T20:44:00.065ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
182026-07-22T20:44:00.070ZWARNtfportd: tfportqsfp0_0 has no addrs
192026-07-22T20:44:00.617ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
202026-07-22T20:44:00.622ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
212026-07-22T20:44:00.626ZWARNtfportd: tfportqsfp0_0 has no addrs
22 [ Jul 22 20:44:00 Rereading configuration. ]
23 [ Jul 22 20:44:00 Executing refresh method (:kill). ]
24 [ Jul 22 20:44:00 Stopping because all processes in service exited. ]
25 [ Jul 22 20:44:00 Executing stop method (:kill). ]
26 [ Jul 22 20:44:00 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
27 [ Jul 22 20:44:00 Method "start" exited with status 0. ]
282026-07-22T20:44:00.716ZINFOtfportd: connecting to dpd host = [::1] port = 12224
292026-07-22T20:44:00.717ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
302026-07-22T20:44:00.717ZINFOtfportd: running in sync-only mode
312026-07-22T20:44:00.717ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
322026-07-22T20:44:00.718ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
332026-07-22T20:44:00.720ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 4d2618f7-a654-441e-b117-49ad2b7cc819, sled_id: fe94a6c0-d682-4570-be03-b85617774e5b, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 1096977e-bf6d-4e17-abce-fdc3e31f9699, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
342026-07-22T20:44:00.720ZDEBGtfportd: created producer ID producer_id = a3242949-cfbb-4f79-a4da-0c38df1410db unit = oximeter
352026-07-22T20:44:00.721ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 4d2618f7-a654-441e-b117-49ad2b7cc819, ), sled_id: Some( fe94a6c0-d682-4570-be03-b85617774e5b, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
362026-07-22T20:44:00.728ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
372026-07-22T20:44:00.730ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::1c8f:b8ff:feb0:1a04->1e:8f:b8:b0:1a:04 interface = tfportrear0_0
382026-07-22T20:44:00.730ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::ec87:55ff:fe31:3f13->ee:87:55:31:3f:13 interface = tfportrear0_0
392026-07-22T20:44:00.732ZDEBGtfportd: registered DTrace probes unit = producer-server
402026-07-22T20:44:00.733ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:59608 method = GET path = /{producer_id} unit = producer-server versions = all
412026-07-22T20:44:00.733ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:59608 unit = producer-server
422026-07-22T20:44:00.733ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:59608 unit = producer-server
432026-07-22T20:44:00.733ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:59608 unit = producer-server
442026-07-22T20:44:00.733ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
452026-07-22T20:44:00.733ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
462026-07-22T20:44:00.733ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
472026-07-22T20:44:00.733ZDEBGtfportd: starting producer registration task unit = producer-server
482026-07-22T20:44:00.734ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:59608 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = a3242949-cfbb-4f79-a4da-0c38df1410db unit = producer-server
492026-07-22T20:44:00.734ZINFOtfportd: created oximeter producer server unit = oximeter
502026-07-22T20:44:00.734ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
512026-07-22T20:44:00.734ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
522026-07-22T20:44:00.735ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 1.761951ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
532026-07-22T20:44:00.735ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-07-22T20:43:43.814357681Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
542026-07-22T20:44:00.735ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 6754314526165846094 unit = oximeter
552026-07-22T20:44:00.735ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
562026-07-22T20:44:00.737ZWARNtfportd: tfportqsfp0_0 has no addrs
572026-07-22T20:44:01.037ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 303.922731ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
582026-07-22T20:44:01.301ZINFOtfportd: added new ARP entry entry = 10.151.1.174->d6:96:dc:ce:4e:a3 interface = tfportqsfp0_0
592026-07-22T20:44:01.554ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 820.400585ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
602026-07-22T20:44:01.998ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
612026-07-22T20:44:02.003ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
622026-07-22T20:44:02.005ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
632026-07-22T20:44:02.006ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
642026-07-22T20:44:02.006ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-07-22T20:43:43.814197950Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
652026-07-22T20:44:02.007ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 8641440025108236958 unit = oximeter
662026-07-22T20:44:02.007ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
672026-07-22T20:44:02.458ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 1.724303255s error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
682026-07-22T20:44:02.735ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
692026-07-22T20:44:02.741ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
702026-07-22T20:44:02.744ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
712026-07-22T20:44:03.680ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("4af4282b-be01-49f6-8049-3debc6d725d0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("596f78be-d601-4a26-9e14-4f14a741a392.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e55d18ea-d658-4064-9ad9-9c89dbdafd93.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e55d18ea-d658-4064-9ad9-9c89dbdafd93.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 718, tv_nsec: 676050620 } }) unit = producer-server
722026-07-22T20:44:03.680ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
732026-07-22T20:44:03.696ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
742026-07-22T20:44:03.742ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
752026-07-22T20:44:03.748ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
762026-07-22T20:44:03.751ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
772026-07-22T20:44:05.155ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
782026-07-22T20:44:05.161ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
792026-07-22T20:44:05.164ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
802026-07-22T20:44:06.383ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
812026-07-22T20:44:06.389ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
822026-07-22T20:44:06.392ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
832026-07-22T20:44:07.470ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
842026-07-22T20:44:07.475ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
852026-07-22T20:44:07.478ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
862026-07-22T20:44:08.677ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
872026-07-22T20:44:08.682ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
882026-07-22T20:44:08.685ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
892026-07-22T20:44:10.098ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902026-07-22T20:44:10.103ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
912026-07-22T20:44:10.107ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
922026-07-22T20:44:10.738ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6754314526165846094 n_samples = 6 unit = oximeter
932026-07-22T20:44:10.767ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942026-07-22T20:44:10.772ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
952026-07-22T20:44:10.775ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
962026-07-22T20:44:12.008ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8641440025108236958 n_samples = 6 unit = oximeter
972026-07-22T20:44:12.198ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
982026-07-22T20:44:12.203ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
992026-07-22T20:44:12.206ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1002026-07-22T20:44:13.105ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1012026-07-22T20:44:13.110ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1022026-07-22T20:44:13.113ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1032026-07-22T20:44:14.344ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1042026-07-22T20:44:14.349ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1052026-07-22T20:44:14.353ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1062026-07-22T20:44:15.393ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1072026-07-22T20:44:15.398ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1082026-07-22T20:44:15.401ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1092026-07-22T20:44:16.087ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1102026-07-22T20:44:16.092ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1112026-07-22T20:44:16.095ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1122026-07-22T20:44:17.348ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1132026-07-22T20:44:17.354ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1142026-07-22T20:44:17.358ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1152026-07-22T20:44:18.698ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1162026-07-22T20:44:18.699ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 17.964985195s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1172026-07-22T20:44:18.704ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1182026-07-22T20:44:18.709ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1192026-07-22T20:44:18.712ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1202026-07-22T20:44:19.744ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1212026-07-22T20:44:19.749ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1222026-07-22T20:44:19.752ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1232026-07-22T20:44:20.745ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6754314526165846094 n_samples = 6 unit = oximeter
1242026-07-22T20:44:20.949ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1252026-07-22T20:44:20.954ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1262026-07-22T20:44:20.957ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1272026-07-22T20:44:22.009ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("4af4282b-be01-49f6-8049-3debc6d725d0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("596f78be-d601-4a26-9e14-4f14a741a392.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e55d18ea-d658-4064-9ad9-9c89dbdafd93.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e55d18ea-d658-4064-9ad9-9c89dbdafd93.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 736, tv_nsec: 997853523 } }) unit = producer-server
1282026-07-22T20:44:22.010ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1292026-07-22T20:44:22.018ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1302026-07-22T20:44:22.018ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8641440025108236958 n_samples = 6 unit = oximeter
1312026-07-22T20:44:22.454ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1322026-07-22T20:44:22.458ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1332026-07-22T20:44:22.461ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1342026-07-22T20:44:23.417ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1352026-07-22T20:44:23.423ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1362026-07-22T20:44:23.426ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1372026-07-22T20:44:24.662ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1382026-07-22T20:44:24.666ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1392026-07-22T20:44:24.669ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1402026-07-22T20:44:25.219ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1412026-07-22T20:44:25.224ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1422026-07-22T20:44:25.227ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1432026-07-22T20:44:25.905ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1442026-07-22T20:44:25.913ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1452026-07-22T20:44:25.917ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1462026-07-22T20:44:26.535ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1472026-07-22T20:44:26.543ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1482026-07-22T20:44:26.548ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1492026-07-22T20:44:27.462ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1502026-07-22T20:44:27.467ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1512026-07-22T20:44:27.471ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1522026-07-22T20:44:28.845ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1532026-07-22T20:44:28.853ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1542026-07-22T20:44:28.859ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1552026-07-22T20:44:29.472ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1562026-07-22T20:44:29.491ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1572026-07-22T20:44:29.495ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1582026-07-22T20:44:30.830ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6754314526165846094 n_samples = 6 unit = oximeter
1592026-07-22T20:44:31.085ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1602026-07-22T20:44:31.102ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1612026-07-22T20:44:31.112ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1622026-07-22T20:44:32.048ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8641440025108236958 n_samples = 6 unit = oximeter
1632026-07-22T20:44:32.445ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1642026-07-22T20:44:32.450ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1652026-07-22T20:44:32.453ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1662026-07-22T20:44:33.767ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1672026-07-22T20:44:33.773ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1682026-07-22T20:44:33.777ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1692026-07-22T20:44:35.241ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1702026-07-22T20:44:35.245ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1712026-07-22T20:44:35.249ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1722026-07-22T20:44:36.101ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1732026-07-22T20:44:36.106ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1742026-07-22T20:44:36.109ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1752026-07-22T20:44:36.791ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1762026-07-22T20:44:36.797ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1772026-07-22T20:44:36.800ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1782026-07-22T20:44:37.060ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1792026-07-22T20:44:37.061ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 36.277155306s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1802026-07-22T20:44:37.727ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1812026-07-22T20:44:37.732ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1822026-07-22T20:44:37.736ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1832026-07-22T20:44:38.995ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1842026-07-22T20:44:39.001ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1852026-07-22T20:44:39.004ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1862026-07-22T20:44:39.938ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1872026-07-22T20:44:39.944ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1882026-07-22T20:44:39.949ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1892026-07-22T20:44:40.647ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1902026-07-22T20:44:40.654ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1912026-07-22T20:44:40.657ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1922026-07-22T20:44:40.853ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6754314526165846094 n_samples = 6 unit = oximeter
1932026-07-22T20:44:41.675ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1942026-07-22T20:44:41.681ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1952026-07-22T20:44:41.684ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1962026-07-22T20:44:42.077ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8641440025108236958 n_samples = 6 unit = oximeter
1972026-07-22T20:44:43.130ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1982026-07-22T20:44:43.134ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
1992026-07-22T20:44:43.137ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2002026-07-22T20:44:43.905ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2012026-07-22T20:44:43.910ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2022026-07-22T20:44:43.914ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2032026-07-22T20:44:44.689ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2042026-07-22T20:44:44.694ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2052026-07-22T20:44:44.697ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2062026-07-22T20:44:44.895ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("4af4282b-be01-49f6-8049-3debc6d725d0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("596f78be-d601-4a26-9e14-4f14a741a392.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e55d18ea-d658-4064-9ad9-9c89dbdafd93.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e55d18ea-d658-4064-9ad9-9c89dbdafd93.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 759, tv_nsec: 825914264 } }) unit = producer-server
2072026-07-22T20:44:44.895ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2082026-07-22T20:44:44.903ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2092026-07-22T20:44:45.821ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2102026-07-22T20:44:45.825ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2112026-07-22T20:44:45.829ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2122026-07-22T20:44:46.367ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2132026-07-22T20:44:46.371ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2142026-07-22T20:44:46.374ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2152026-07-22T20:44:47.827ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2162026-07-22T20:44:47.833ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2172026-07-22T20:44:47.836ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2182026-07-22T20:44:49.238ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2192026-07-22T20:44:49.243ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2202026-07-22T20:44:49.246ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2212026-07-22T20:44:49.983ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2222026-07-22T20:44:49.990ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2232026-07-22T20:44:49.993ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2242026-07-22T20:44:50.861ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 6754314526165846094 n_samples = 6 unit = oximeter
2252026-07-22T20:44:50.958ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2262026-07-22T20:44:50.966ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2272026-07-22T20:44:50.971ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2282026-07-22T20:44:52.060ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2292026-07-22T20:44:52.067ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2302026-07-22T20:44:52.071ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2312026-07-22T20:44:52.081ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8641440025108236958 n_samples = 6 unit = oximeter
2322026-07-22T20:44:53.578ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2332026-07-22T20:44:53.583ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2342026-07-22T20:44:53.585ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2352026-07-22T20:44:54.539ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2362026-07-22T20:44:54.543ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2372026-07-22T20:44:54.546ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2382026-07-22T20:44:55.775ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2392026-07-22T20:44:55.779ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2402026-07-22T20:44:55.783ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2412026-07-22T20:44:57.115ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2422026-07-22T20:44:57.120ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2432026-07-22T20:44:57.123ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2442026-07-22T20:44:57.928ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2452026-07-22T20:44:57.932ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2462026-07-22T20:44:57.936ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2472026-07-22T20:44:59.279ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2482026-07-22T20:44:59.284ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2492026-07-22T20:44:59.288ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2502026-07-22T20:44:59.903ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2512026-07-22T20:44:59.904ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 59.106435671s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2522026-07-22T20:45:00.419ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2532026-07-22T20:45:00.425ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::ec87:55ff:fe31:3f13]
2542026-07-22T20:45:00.428ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]