Max level shown:
1 [ Aug 20 17:58:07 Enabled. ]
2 [ Aug 20 17:58:07 Rereading configuration. ]
3 [ Aug 20 17:58:08 Rereading configuration. ]
4 [ Aug 20 17:58:19 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Aug 20 17:58:19 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-08-20T17:58:19.218ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-08-20T17:58:19.219ZINFOtfportd: connected to dpd running softnpu-5cb3f22d
92026-08-20T17:58:19.220ZINFOtfportd: running in sync-only mode
102026-08-20T17:58:19.220ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-08-20T17:58:19.222ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-08-20T17:58:19.223ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 0bbad842-be25-4d43-94da-4d625021c11e, ), sled_id: Some( 0fc77c9a-fb99-4741-be87-5d50327faaf1, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
142026-08-20T17:58:19.223ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-08-20T17:58:19.229ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::688d:5eff:fe7f:fa1d->6a:8d:5e:7f:fa:1d interface = tfportrear0_0
162026-08-20T17:58:19.231ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-08-20T17:58:19.238ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
182026-08-20T17:58:19.244ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Aug 20 17:58:19 Rereading configuration. ]
20 [ Aug 20 17:58:19 Executing refresh method (:kill). ]
21 [ Aug 20 17:58:19 Stopping because all processes in service exited. ]
22 [ Aug 20 17:58:19 Executing stop method (:kill). ]
23 [ Aug 20 17:58:19 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Aug 20 17:58:19 Method "start" exited with status 0. ]
252026-08-20T17:58:19.846ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-08-20T17:58:19.847ZINFOtfportd: connected to dpd running softnpu-5cb3f22d
272026-08-20T17:58:19.848ZINFOtfportd: running in sync-only mode
282026-08-20T17:58:19.848ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-08-20T17:58:19.850ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-08-20T17:58:19.850ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 0bbad842-be25-4d43-94da-4d625021c11e, sled_id: 0fc77c9a-fb99-4741-be87-5d50327faaf1, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 02f453a3-d84a-48d8-a1ae-fcedc8eec30e, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-08-20T17:58:19.850ZDEBGtfportd: created producer ID producer_id = 0689f77b-9df6-42da-8bf3-ecdfaab0a8e1 unit = oximeter
322026-08-20T17:58:19.852ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 0bbad842-be25-4d43-94da-4d625021c11e, ), sled_id: Some( 0fc77c9a-fb99-4741-be87-5d50327faaf1, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
332026-08-20T17:58:19.859ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-08-20T17:58:19.861ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::7056:d1ff:fee3:93b7->72:56:d1:e3:93:b7 interface = tfportrear0_0
352026-08-20T17:58:19.861ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::688d:5eff:fe7f:fa1d->6a:8d:5e:7f:fa:1d interface = tfportrear0_0
362026-08-20T17:58:19.863ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-08-20T17:58:19.863ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:51001 method = GET path = /{producer_id} unit = producer-server versions = all
382026-08-20T17:58:19.863ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:51001 unit = producer-server
392026-08-20T17:58:19.864ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:51001 unit = producer-server
402026-08-20T17:58:19.864ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:51001 unit = producer-server
412026-08-20T17:58:19.864ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-08-20T17:58:19.864ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-08-20T17:58:19.864ZDEBGtfportd: starting producer registration task unit = producer-server
442026-08-20T17:58:19.864ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:51001 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 0689f77b-9df6-42da-8bf3-ecdfaab0a8e1 unit = producer-server
452026-08-20T17:58:19.864ZINFOtfportd: created oximeter producer server unit = oximeter
462026-08-20T17:58:19.864ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-08-20T17:58:19.864ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
482026-08-20T17:58:19.865ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-08-20T17:58:19.866ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 1.287616ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
502026-08-20T17:58:19.866ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-08-20T17:58:03.093186568Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
512026-08-20T17:58:19.866ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 8750171205175392197 unit = oximeter
522026-08-20T17:58:19.866ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
532026-08-20T17:58:19.868ZWARNtfportd: tfportqsfp0_0 has no addrs
542026-08-20T17:58:20.166ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 301.565601ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
552026-08-20T17:58:20.662ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 797.510347ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
562026-08-20T17:58:20.962ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
572026-08-20T17:58:20.968ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
582026-08-20T17:58:20.971ZWARNtfportd: tfportqsfp0_0 has no addrs
592026-08-20T17:58:21.352ZINFOtfportd: added new ARP entry entry = 10.151.2.174->d2:67:5b:27:65:4a interface = tfportqsfp0_0
602026-08-20T17:58:21.514ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
612026-08-20T17:58:21.519ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
622026-08-20T17:58:21.522ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
632026-08-20T17:58:21.522ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
642026-08-20T17:58:21.523ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-08-20T17:58:03.093055745Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
652026-08-20T17:58:21.523ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 10405661656293805585 unit = oximeter
662026-08-20T17:58:21.523ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
672026-08-20T17:58:22.154ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9388cf0a-e8f6-48d3-94be-288aeacc45d1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6d96ce8-753b-4b91-992b-565812139713.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f6da1a6a-c822-49c1-af56-850083035003.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f6da1a6a-c822-49c1-af56-850083035003.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 738, tv_nsec: 266774024 } }) unit = producer-server
682026-08-20T17:58:22.154ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
692026-08-20T17:58:22.171ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
702026-08-20T17:58:22.360ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-08-20T17:58:22.365ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
722026-08-20T17:58:22.368ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
732026-08-20T17:58:23.183ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-08-20T17:58:23.188ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
752026-08-20T17:58:23.191ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
762026-08-20T17:58:24.660ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-08-20T17:58:24.665ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
782026-08-20T17:58:24.668ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
792026-08-20T17:58:25.288ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-08-20T17:58:25.293ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
812026-08-20T17:58:25.296ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
822026-08-20T17:58:26.802ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-08-20T17:58:26.807ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
842026-08-20T17:58:26.809ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
852026-08-20T17:58:28.191ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862026-08-20T17:58:28.196ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
872026-08-20T17:58:28.199ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
882026-08-20T17:58:29.251ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
892026-08-20T17:58:29.255ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
902026-08-20T17:58:29.258ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
912026-08-20T17:58:29.857ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
922026-08-20T17:58:29.862ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
932026-08-20T17:58:29.865ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
942026-08-20T17:58:29.871ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8750171205175392197 n_samples = 6 unit = oximeter
952026-08-20T17:58:30.718ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
962026-08-20T17:58:30.724ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
972026-08-20T17:58:30.727ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
982026-08-20T17:58:31.303ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
992026-08-20T17:58:31.308ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1002026-08-20T17:58:31.312ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1012026-08-20T17:58:31.524ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10405661656293805585 n_samples = 6 unit = oximeter
1022026-08-20T17:58:32.230ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-08-20T17:58:32.234ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1042026-08-20T17:58:32.238ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1052026-08-20T17:58:33.348ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-08-20T17:58:33.353ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1072026-08-20T17:58:33.356ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1082026-08-20T17:58:34.387ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-08-20T17:58:34.391ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1102026-08-20T17:58:34.395ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1112026-08-20T17:58:35.022ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-08-20T17:58:35.027ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1132026-08-20T17:58:35.030ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1142026-08-20T17:58:36.484ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-08-20T17:58:36.489ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1162026-08-20T17:58:36.493ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1172026-08-20T17:58:37.064ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1182026-08-20T17:58:37.065ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 17.309284266s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1192026-08-20T17:58:37.275ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1202026-08-20T17:58:37.280ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1212026-08-20T17:58:37.283ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1222026-08-20T17:58:38.626ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1232026-08-20T17:58:38.631ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1242026-08-20T17:58:38.635ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1252026-08-20T17:58:39.366ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1262026-08-20T17:58:39.372ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1272026-08-20T17:58:39.375ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1282026-08-20T17:58:39.759ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8750171205175392197 n_samples = 6 unit = oximeter
1292026-08-20T17:58:40.068ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9388cf0a-e8f6-48d3-94be-288aeacc45d1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6d96ce8-753b-4b91-992b-565812139713.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f6da1a6a-c822-49c1-af56-850083035003.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f6da1a6a-c822-49c1-af56-850083035003.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 756, tv_nsec: 295930754 } }) unit = producer-server
1302026-08-20T17:58:40.068ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1312026-08-20T17:58:40.081ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1322026-08-20T17:58:40.741ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1332026-08-20T17:58:40.747ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1342026-08-20T17:58:40.750ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1352026-08-20T17:58:41.410ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10405661656293805585 n_samples = 6 unit = oximeter
1362026-08-20T17:58:42.139ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-08-20T17:58:42.145ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1382026-08-20T17:58:42.148ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1392026-08-20T17:58:43.014ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-08-20T17:58:43.021ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1412026-08-20T17:58:43.025ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1422026-08-20T17:58:43.575ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-08-20T17:58:43.581ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1442026-08-20T17:58:43.585ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1452026-08-20T17:58:45.032ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-08-20T17:58:45.042ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1472026-08-20T17:58:45.046ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1482026-08-20T17:58:46.110ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-08-20T17:58:46.116ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1502026-08-20T17:58:46.121ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1512026-08-20T17:58:47.179ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-08-20T17:58:47.185ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1532026-08-20T17:58:47.187ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1542026-08-20T17:58:47.882ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1552026-08-20T17:58:47.889ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1562026-08-20T17:58:47.893ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1572026-08-20T17:58:48.852ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1582026-08-20T17:58:48.856ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1592026-08-20T17:58:48.860ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1602026-08-20T17:58:49.759ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8750171205175392197 n_samples = 6 unit = oximeter
1612026-08-20T17:58:49.999ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1622026-08-20T17:58:50.005ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1632026-08-20T17:58:50.008ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1642026-08-20T17:58:50.811ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1652026-08-20T17:58:50.816ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1662026-08-20T17:58:50.819ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1672026-08-20T17:58:51.414ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10405661656293805585 n_samples = 6 unit = oximeter
1682026-08-20T17:58:51.444ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1692026-08-20T17:58:51.449ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1702026-08-20T17:58:51.453ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1712026-08-20T17:58:52.586ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-08-20T17:58:52.590ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1732026-08-20T17:58:52.593ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1742026-08-20T17:58:53.272ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1752026-08-20T17:58:53.277ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1762026-08-20T17:58:53.280ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1772026-08-20T17:58:54.601ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1782026-08-20T17:58:54.608ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1792026-08-20T17:58:54.612ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1802026-08-20T17:58:55.088ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1812026-08-20T17:58:55.089ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 35.340358985s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1822026-08-20T17:58:55.399ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1832026-08-20T17:58:55.405ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1842026-08-20T17:58:55.409ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1852026-08-20T17:58:56.756ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862026-08-20T17:58:56.761ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1872026-08-20T17:58:56.765ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1882026-08-20T17:58:57.761ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-08-20T17:58:57.766ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1902026-08-20T17:58:57.770ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1912026-08-20T17:58:58.747ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1922026-08-20T17:58:58.752ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1932026-08-20T17:58:58.755ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1942026-08-20T17:58:59.467ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9388cf0a-e8f6-48d3-94be-288aeacc45d1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6d96ce8-753b-4b91-992b-565812139713.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f6da1a6a-c822-49c1-af56-850083035003.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f6da1a6a-c822-49c1-af56-850083035003.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 775, tv_nsec: 695606982 } }) unit = producer-server
1952026-08-20T17:58:59.468ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1962026-08-20T17:58:59.475ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1972026-08-20T17:58:59.677ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1982026-08-20T17:58:59.682ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
1992026-08-20T17:58:59.684ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2002026-08-20T17:58:59.764ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8750171205175392197 n_samples = 6 unit = oximeter
2012026-08-20T17:59:00.252ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2022026-08-20T17:59:00.256ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2032026-08-20T17:59:00.259ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2042026-08-20T17:59:00.998ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2052026-08-20T17:59:01.003ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2062026-08-20T17:59:01.006ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2072026-08-20T17:59:01.418ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10405661656293805585 n_samples = 6 unit = oximeter
2082026-08-20T17:59:01.627ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-08-20T17:59:01.634ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2102026-08-20T17:59:01.638ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2112026-08-20T17:59:02.766ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-08-20T17:59:02.771ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2132026-08-20T17:59:02.774ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2142026-08-20T17:59:04.297ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2152026-08-20T17:59:05.752ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2162026-08-20T17:59:05.756ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2172026-08-20T17:59:06.554ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2182026-08-20T17:59:06.560ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2192026-08-20T17:59:06.565ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2202026-08-20T17:59:07.998ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2212026-08-20T17:59:08.007ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2222026-08-20T17:59:08.011ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2232026-08-20T17:59:09.381ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2242026-08-20T17:59:09.389ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2252026-08-20T17:59:09.394ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2262026-08-20T17:59:09.768ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8750171205175392197 n_samples = 6 unit = oximeter
2272026-08-20T17:59:10.039ZINFOtfportd: removed stale NDP entry :fe80::7056:d1ff:fee3:93b7->72:56:d1:e3:93:b7
2282026-08-20T17:59:10.788ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2292026-08-20T17:59:10.796ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2302026-08-20T17:59:10.801ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2312026-08-20T17:59:11.199ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::7056:d1ff:fee3:93b7->72:56:d1:e3:93:b7 interface = tfportrear0_0
2322026-08-20T17:59:11.422ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10405661656293805585 n_samples = 6 unit = oximeter
2332026-08-20T17:59:11.548ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-08-20T17:59:11.565ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2352026-08-20T17:59:11.570ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2362026-08-20T17:59:12.418ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372026-08-20T17:59:12.438ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2382026-08-20T17:59:12.451ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2392026-08-20T17:59:14.006ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402026-08-20T17:59:14.016ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2412026-08-20T17:59:14.021ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2422026-08-20T17:59:14.492ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2432026-08-20T17:59:14.554ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 54.801499078s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2442026-08-20T17:59:15.121ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2452026-08-20T17:59:15.132ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2462026-08-20T17:59:15.143ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2472026-08-20T17:59:19.989ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2482026-08-20T17:59:20.033ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2492026-08-20T17:59:20.047ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2502026-08-20T17:59:20.208ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8750171205175392197 n_samples = 6 unit = oximeter
2512026-08-20T17:59:20.888ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9388cf0a-e8f6-48d3-94be-288aeacc45d1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6d96ce8-753b-4b91-992b-565812139713.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f6da1a6a-c822-49c1-af56-850083035003.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f6da1a6a-c822-49c1-af56-850083035003.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 797, tv_nsec: 114545558 } }) unit = producer-server
2522026-08-20T17:59:20.889ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
2532026-08-20T17:59:20.929ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
2542026-08-20T17:59:21.425ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10405661656293805585 n_samples = 6 unit = oximeter
2552026-08-20T17:59:21.468ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2562026-08-20T17:59:21.476ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2572026-08-20T17:59:21.480ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2582026-08-20T17:59:22.517ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2592026-08-20T17:59:22.544ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2602026-08-20T17:59:22.599ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2612026-08-20T17:59:23.238ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2622026-08-20T17:59:23.246ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2632026-08-20T17:59:23.251ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2642026-08-20T17:59:24.473ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2652026-08-20T17:59:24.484ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2662026-08-20T17:59:24.490ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2672026-08-20T17:59:25.645ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2682026-08-20T17:59:25.655ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2692026-08-20T17:59:25.660ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2702026-08-20T17:59:26.908ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2712026-08-20T17:59:26.916ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2722026-08-20T17:59:26.921ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2732026-08-20T17:59:28.022ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2742026-08-20T17:59:28.032ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2752026-08-20T17:59:28.038ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2762026-08-20T17:59:29.122ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2772026-08-20T17:59:29.130ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2782026-08-20T17:59:29.134ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2792026-08-20T17:59:29.698ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2802026-08-20T17:59:29.705ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2812026-08-20T17:59:29.710ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2822026-08-20T17:59:30.212ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8750171205175392197 n_samples = 6 unit = oximeter
2832026-08-20T17:59:30.902ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2842026-08-20T17:59:30.918ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2852026-08-20T17:59:30.928ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2862026-08-20T17:59:31.432ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10405661656293805585 n_samples = 6 unit = oximeter
2872026-08-20T17:59:31.795ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2882026-08-20T17:59:31.803ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2892026-08-20T17:59:31.808ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2902026-08-20T17:59:32.548ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2912026-08-20T17:59:32.560ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2922026-08-20T17:59:32.565ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2932026-08-20T17:59:33.400ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2942026-08-20T17:59:33.409ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2952026-08-20T17:59:33.412ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2962026-08-20T17:59:34.423ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2972026-08-20T17:59:34.432ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
2982026-08-20T17:59:34.437ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2992026-08-20T17:59:35.241ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3002026-08-20T17:59:35.251ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3012026-08-20T17:59:35.255ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3022026-08-20T17:59:35.937ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3032026-08-20T17:59:35.943ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 76.193805488s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3042026-08-20T17:59:36.098ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3052026-08-20T17:59:36.107ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3062026-08-20T17:59:36.112ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3072026-08-20T17:59:37.207ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3082026-08-20T17:59:37.217ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3092026-08-20T17:59:37.222ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3102026-08-20T17:59:38.477ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3112026-08-20T17:59:38.485ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3122026-08-20T17:59:38.490ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3132026-08-20T17:59:39.711ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3142026-08-20T17:59:39.719ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3152026-08-20T17:59:39.723ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3162026-08-20T17:59:40.218ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8750171205175392197 n_samples = 6 unit = oximeter
3172026-08-20T17:59:40.366ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3182026-08-20T17:59:40.376ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3192026-08-20T17:59:40.381ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3202026-08-20T17:59:41.192ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3212026-08-20T17:59:41.199ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3222026-08-20T17:59:41.203ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3232026-08-20T17:59:41.438ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10405661656293805585 n_samples = 6 unit = oximeter
3242026-08-20T17:59:42.505ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3252026-08-20T17:59:42.511ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3262026-08-20T17:59:42.514ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3272026-08-20T17:59:43.416ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3282026-08-20T17:59:43.422ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3292026-08-20T17:59:43.426ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3302026-08-20T17:59:44.708ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3312026-08-20T17:59:44.715ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3322026-08-20T17:59:44.718ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3332026-08-20T17:59:45.241ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3342026-08-20T17:59:45.247ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3352026-08-20T17:59:45.251ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3362026-08-20T17:59:46.382ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3372026-08-20T17:59:46.389ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3382026-08-20T17:59:46.393ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3392026-08-20T17:59:47.283ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3402026-08-20T17:59:47.295ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3412026-08-20T17:59:47.299ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3422026-08-20T17:59:48.644ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3432026-08-20T17:59:48.650ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3442026-08-20T17:59:48.654ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3452026-08-20T17:59:49.778ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3462026-08-20T17:59:49.786ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3472026-08-20T17:59:49.790ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3482026-08-20T17:59:50.223ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8750171205175392197 n_samples = 6 unit = oximeter
3492026-08-20T17:59:51.267ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3502026-08-20T17:59:51.273ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3512026-08-20T17:59:51.276ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3522026-08-20T17:59:51.439ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10405661656293805585 n_samples = 6 unit = oximeter
3532026-08-20T17:59:52.637ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3542026-08-20T17:59:52.643ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3552026-08-20T17:59:52.647ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3562026-08-20T17:59:53.210ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9388cf0a-e8f6-48d3-94be-288aeacc45d1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d6d96ce8-753b-4b91-992b-565812139713.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("f6da1a6a-c822-49c1-af56-850083035003.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f6da1a6a-c822-49c1-af56-850083035003.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 829, tv_nsec: 439139059 } }) unit = producer-server
3572026-08-20T17:59:53.211ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
3582026-08-20T17:59:53.218ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
3592026-08-20T17:59:53.227ZDEBGtfportd (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "816bfb42-26db-49ab-a2ef-7ccb31c0fd4f", "content-length": "41", "date": "Thu, 20 Aug 2026 17:59:53 GMT"} }) unit = producer-server
3602026-08-20T17:59:53.229ZDEBGtfportd (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s unit = producer-server wait_period = 150s
3612026-08-20T17:59:53.530ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3622026-08-20T17:59:53.534ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3632026-08-20T17:59:53.538ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3642026-08-20T17:59:54.583ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3652026-08-20T17:59:54.589ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3662026-08-20T17:59:54.592ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3672026-08-20T17:59:55.345ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3682026-08-20T17:59:55.351ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3692026-08-20T17:59:55.354ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3702026-08-20T17:59:56.609ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3712026-08-20T17:59:56.621ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3722026-08-20T17:59:56.624ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3732026-08-20T17:59:57.210ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3742026-08-20T17:59:57.215ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3752026-08-20T17:59:57.219ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3762026-08-20T17:59:58.065ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3772026-08-20T17:59:58.071ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3782026-08-20T17:59:58.079ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3792026-08-20T17:59:58.790ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3802026-08-20T17:59:58.796ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3812026-08-20T17:59:58.799ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3822026-08-20T17:59:59.967ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3832026-08-20T17:59:59.972ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3842026-08-20T17:59:59.975ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3852026-08-20T18:00:00.227ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8750171205175392197 n_samples = 6 unit = oximeter
3862026-08-20T18:00:00.984ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3872026-08-20T18:00:00.993ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3882026-08-20T18:00:00.999ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3892026-08-20T18:00:01.441ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10405661656293805585 n_samples = 6 unit = oximeter
3902026-08-20T18:00:02.253ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3912026-08-20T18:00:02.261ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::688d:5eff:fe7f:fa1d]
3922026-08-20T18:00:02.267ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]