Max level shown:
1 [ Jul 24 07:28:55 Enabled. ]
2 [ Jul 24 07:28:55 Rereading configuration. ]
3 [ Jul 24 07:28:56 Rereading configuration. ]
4 [ Jul 24 07:29:07 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jul 24 07:29:07 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-07-24T07:29:07.278ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-07-24T07:29:07.304ZINFOtfportd: connected to dpd running softnpu-5cb3f22d
92026-07-24T07:29:07.304ZINFOtfportd: running in sync-only mode
102026-07-24T07:29:07.304ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-07-24T07:29:07.306ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-07-24T07:29:07.309ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 7d9cfec8-8b0d-48d9-8b33-92444f0217b5, ), sled_id: Some( 855dfc9c-0f28-4206-8894-587cebdfe6a5, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-07-24T07:29:07.309ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-07-24T07:29:07.313ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::8073:c2ff:fec6:5446->82:73:c2:c6:54:46 interface = tfportrear0_0
162026-07-24T07:29:07.313ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::b0d5:d3ff:fe49:bccf->b2:d5:d3:49:bc:cf interface = tfportrear0_0
172026-07-24T07:29:07.314ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
182026-07-24T07:29:07.320ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
192026-07-24T07:29:07.325ZWARNtfportd: tfportqsfp0_0 has no addrs
20 [ Jul 24 07:29:07 Rereading configuration. ]
21 [ Jul 24 07:29:07 Executing refresh method (:kill). ]
22 [ Jul 24 07:29:07 Stopping because all processes in service exited. ]
23 [ Jul 24 07:29:07 Executing stop method (:kill). ]
24 [ Jul 24 07:29:07 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
25 [ Jul 24 07:29:07 Method "start" exited with status 0. ]
262026-07-24T07:29:07.995ZINFOtfportd: connecting to dpd host = [::1] port = 12224
272026-07-24T07:29:07.995ZINFOtfportd: connected to dpd running softnpu-5cb3f22d
282026-07-24T07:29:07.996ZINFOtfportd: running in sync-only mode
292026-07-24T07:29:07.996ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
302026-07-24T07:29:07.998ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
312026-07-24T07:29:07.998ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 7d9cfec8-8b0d-48d9-8b33-92444f0217b5, sled_id: 855dfc9c-0f28-4206-8894-587cebdfe6a5, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: a5363dad-5856-4893-ba65-1a9c3201afd8, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
322026-07-24T07:29:07.998ZDEBGtfportd: created producer ID producer_id = 5df73ff9-dc73-424c-b9c2-d0a496dc5ec8 unit = oximeter
332026-07-24T07:29:07.999ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 7d9cfec8-8b0d-48d9-8b33-92444f0217b5, ), sled_id: Some( 855dfc9c-0f28-4206-8894-587cebdfe6a5, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
342026-07-24T07:29:08.006ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
352026-07-24T07:29:08.008ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::8073:c2ff:fec6:5446->82:73:c2:c6:54:46 interface = tfportrear0_0
362026-07-24T07:29:08.009ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::b0d5:d3ff:fe49:bccf->b2:d5:d3:49:bc:cf interface = tfportrear0_0
372026-07-24T07:29:08.009ZDEBGtfportd: registered DTrace probes unit = producer-server
382026-07-24T07:29:08.010ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:46814 method = GET path = /{producer_id} unit = producer-server versions = all
392026-07-24T07:29:08.010ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:46814 unit = producer-server
402026-07-24T07:29:08.011ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:46814 unit = producer-server
412026-07-24T07:29:08.011ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:46814 unit = producer-server
422026-07-24T07:29:08.011ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
432026-07-24T07:29:08.011ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
442026-07-24T07:29:08.011ZDEBGtfportd: starting producer registration task unit = producer-server
452026-07-24T07:29:08.011ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:46814 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 5df73ff9-dc73-424c-b9c2-d0a496dc5ec8 unit = producer-server
462026-07-24T07:29:08.011ZINFOtfportd: created oximeter producer server unit = oximeter
472026-07-24T07:29:08.011ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
482026-07-24T07:29:08.012ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
492026-07-24T07:29:08.012ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
502026-07-24T07:29:08.013ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 1.697271ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
512026-07-24T07:29:08.014ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-07-24T07:28:51.056881836Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
522026-07-24T07:29:08.014ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 3657787801977654492 unit = oximeter
532026-07-24T07:29:08.014ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
542026-07-24T07:29:08.015ZWARNtfportd: tfportqsfp0_0 has no addrs
552026-07-24T07:29:08.199ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 187.788831ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
562026-07-24T07:29:08.529ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 517.497742ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
572026-07-24T07:29:09.106ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
582026-07-24T07:29:09.111ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
592026-07-24T07:29:09.115ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
602026-07-24T07:29:09.115ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
612026-07-24T07:29:09.116ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-07-24T07:28:51.056751712Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
622026-07-24T07:29:09.116ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 16470991053604711316 unit = oximeter
632026-07-24T07:29:09.117ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
642026-07-24T07:29:09.278ZINFOtfportd: added new ARP entry entry = 10.151.1.174->16:f1:67:2a:2e:d3 interface = tfportqsfp0_0
652026-07-24T07:29:09.291ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("107fa693-78f9-4bb0-8551-cc9e3f208fad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("13ac4e4d-4bcd-4ef6-9d15-7cfa916a0627.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2cf63006-532d-4ca5-8db7-8ec0f1839958.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("2cf63006-532d-4ca5-8db7-8ec0f1839958.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 999, tv_nsec: 851412463 } }) unit = producer-server
662026-07-24T07:29:09.292ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
672026-07-24T07:29:09.309ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
682026-07-24T07:29:10.056ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
692026-07-24T07:29:10.061ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
702026-07-24T07:29:10.063ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
712026-07-24T07:29:11.392ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
722026-07-24T07:29:11.397ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
732026-07-24T07:29:11.400ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
742026-07-24T07:29:12.264ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
752026-07-24T07:29:12.270ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
762026-07-24T07:29:12.273ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
772026-07-24T07:29:12.906ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
782026-07-24T07:29:12.911ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
792026-07-24T07:29:12.915ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
802026-07-24T07:29:14.093ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
812026-07-24T07:29:14.098ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
822026-07-24T07:29:14.112ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
832026-07-24T07:29:15.206ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
842026-07-24T07:29:15.212ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
852026-07-24T07:29:15.215ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
862026-07-24T07:29:15.853ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
872026-07-24T07:29:15.857ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
882026-07-24T07:29:15.860ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
892026-07-24T07:29:16.458ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902026-07-24T07:29:16.463ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
912026-07-24T07:29:16.465ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
922026-07-24T07:29:17.888ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
932026-07-24T07:29:17.892ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
942026-07-24T07:29:17.896ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
952026-07-24T07:29:18.017ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3657787801977654492 n_samples = 6 unit = oximeter
962026-07-24T07:29:18.451ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-07-24T07:29:18.456ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
982026-07-24T07:29:18.459ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
992026-07-24T07:29:19.120ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16470991053604711316 n_samples = 6 unit = oximeter
1002026-07-24T07:29:19.725ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1012026-07-24T07:29:19.730ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1022026-07-24T07:29:19.734ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1032026-07-24T07:29:20.487ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1042026-07-24T07:29:20.492ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1052026-07-24T07:29:20.495ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1062026-07-24T07:29:21.132ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1072026-07-24T07:29:21.138ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1082026-07-24T07:29:21.142ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1092026-07-24T07:29:22.003ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1102026-07-24T07:29:22.008ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1112026-07-24T07:29:22.011ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1122026-07-24T07:29:22.678ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1132026-07-24T07:29:22.683ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1142026-07-24T07:29:22.686ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1152026-07-24T07:29:23.925ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1162026-07-24T07:29:23.929ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1172026-07-24T07:29:23.932ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1182026-07-24T07:29:24.311ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1192026-07-24T07:29:24.312ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 16.300509732s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1202026-07-24T07:29:24.743ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1212026-07-24T07:29:24.749ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1222026-07-24T07:29:24.752ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1232026-07-24T07:29:26.310ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1242026-07-24T07:29:26.315ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1252026-07-24T07:29:26.318ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1262026-07-24T07:29:26.828ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1272026-07-24T07:29:26.833ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1282026-07-24T07:29:26.836ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1292026-07-24T07:29:27.343ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("107fa693-78f9-4bb0-8551-cc9e3f208fad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("13ac4e4d-4bcd-4ef6-9d15-7cfa916a0627.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2cf63006-532d-4ca5-8db7-8ec0f1839958.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("2cf63006-532d-4ca5-8db7-8ec0f1839958.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 1017, tv_nsec: 374934035 } }) unit = producer-server
1302026-07-24T07:29:27.343ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1312026-07-24T07:29:27.351ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1322026-07-24T07:29:28.316ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1332026-07-24T07:29:28.321ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1342026-07-24T07:29:28.324ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1352026-07-24T07:29:28.547ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3657787801977654492 n_samples = 6 unit = oximeter
1362026-07-24T07:29:29.351ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-07-24T07:29:29.357ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1382026-07-24T07:29:29.360ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1392026-07-24T07:29:29.649ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16470991053604711316 n_samples = 6 unit = oximeter
1402026-07-24T07:29:30.208ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1412026-07-24T07:29:30.213ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1422026-07-24T07:29:30.217ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1432026-07-24T07:29:30.897ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1442026-07-24T07:29:30.902ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1452026-07-24T07:29:30.905ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1462026-07-24T07:29:31.818ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1472026-07-24T07:29:31.823ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1482026-07-24T07:29:31.826ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1492026-07-24T07:29:32.485ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1502026-07-24T07:29:32.491ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1512026-07-24T07:29:32.494ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1522026-07-24T07:29:33.248ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1532026-07-24T07:29:33.253ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1542026-07-24T07:29:33.256ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1552026-07-24T07:29:34.214ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1562026-07-24T07:29:34.220ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1572026-07-24T07:29:34.223ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1582026-07-24T07:29:34.932ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1592026-07-24T07:29:34.936ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1602026-07-24T07:29:34.939ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1612026-07-24T07:29:36.027ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1622026-07-24T07:29:36.032ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1632026-07-24T07:29:36.035ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1642026-07-24T07:29:37.017ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1652026-07-24T07:29:37.022ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1662026-07-24T07:29:37.025ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1672026-07-24T07:29:37.988ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1682026-07-24T07:29:37.994ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1692026-07-24T07:29:37.997ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1702026-07-24T07:29:38.523ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3657787801977654492 n_samples = 6 unit = oximeter
1712026-07-24T07:29:39.383ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-07-24T07:29:39.388ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1732026-07-24T07:29:39.392ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1742026-07-24T07:29:39.623ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16470991053604711316 n_samples = 6 unit = oximeter
1752026-07-24T07:29:40.759ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1762026-07-24T07:29:40.764ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1772026-07-24T07:29:40.767ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1782026-07-24T07:29:41.288ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1792026-07-24T07:29:41.292ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1802026-07-24T07:29:41.295ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1812026-07-24T07:29:42.316ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1822026-07-24T07:29:42.317ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 33.814744353s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1832026-07-24T07:29:42.445ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1842026-07-24T07:29:42.450ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1852026-07-24T07:29:42.453ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1862026-07-24T07:29:43.691ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1872026-07-24T07:29:43.695ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1882026-07-24T07:29:43.698ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1892026-07-24T07:29:44.667ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1902026-07-24T07:29:44.673ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1912026-07-24T07:29:44.676ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1922026-07-24T07:29:44.978ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("107fa693-78f9-4bb0-8551-cc9e3f208fad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("13ac4e4d-4bcd-4ef6-9d15-7cfa916a0627.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2cf63006-532d-4ca5-8db7-8ec0f1839958.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("2cf63006-532d-4ca5-8db7-8ec0f1839958.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 1035, tv_nsec: 54607938 } }) unit = producer-server
1932026-07-24T07:29:44.979ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1942026-07-24T07:29:44.986ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1952026-07-24T07:29:46.099ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1962026-07-24T07:29:46.104ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
1972026-07-24T07:29:46.107ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1982026-07-24T07:29:47.569ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1992026-07-24T07:29:47.574ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2002026-07-24T07:29:47.577ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2012026-07-24T07:29:48.391ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2022026-07-24T07:29:48.396ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2032026-07-24T07:29:48.400ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2042026-07-24T07:29:48.498ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3657787801977654492 n_samples = 6 unit = oximeter
2052026-07-24T07:29:49.598ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16470991053604711316 n_samples = 6 unit = oximeter
2062026-07-24T07:29:49.779ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2072026-07-24T07:29:49.783ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2082026-07-24T07:29:49.787ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2092026-07-24T07:29:51.249ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2102026-07-24T07:29:51.254ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2112026-07-24T07:29:51.258ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2122026-07-24T07:29:52.698ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2132026-07-24T07:29:52.703ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2142026-07-24T07:29:52.706ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2152026-07-24T07:29:54.115ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2162026-07-24T07:29:54.120ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2172026-07-24T07:29:54.123ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2182026-07-24T07:29:54.727ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2192026-07-24T07:29:54.732ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2202026-07-24T07:29:54.735ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2212026-07-24T07:29:55.535ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2222026-07-24T07:29:55.543ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2232026-07-24T07:29:55.547ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2242026-07-24T07:29:56.123ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2252026-07-24T07:29:56.129ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2262026-07-24T07:29:56.133ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2272026-07-24T07:29:56.927ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2282026-07-24T07:29:56.932ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2292026-07-24T07:29:56.936ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2302026-07-24T07:29:57.494ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2312026-07-24T07:29:57.500ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2322026-07-24T07:29:57.503ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2332026-07-24T07:29:58.207ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-07-24T07:29:58.213ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2352026-07-24T07:29:58.217ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2362026-07-24T07:29:58.473ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 3657787801977654492 n_samples = 6 unit = oximeter
2372026-07-24T07:29:59.483ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2382026-07-24T07:29:59.495ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2392026-07-24T07:29:59.500ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2402026-07-24T07:29:59.643ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16470991053604711316 n_samples = 6 unit = oximeter
2412026-07-24T07:29:59.948ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2422026-07-24T07:29:59.950ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 51.494648501s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2432026-07-24T07:30:00.141ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2442026-07-24T07:30:00.179ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2452026-07-24T07:30:00.204ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2462026-07-24T07:30:01.580ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2472026-07-24T07:30:01.606ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::8073:c2ff:fec6:5446]
2482026-07-24T07:30:01.618ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]