Max level shown:
1 [ Feb 5 23:43:02 Enabled. ]
2 [ Feb 5 23:43:02 Rereading configuration. ]
3 [ Feb 5 23:43:03 Rereading configuration. ]
4 [ Feb 5 23:43:14 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Feb 5 23:43:14 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-02-05T23:43:14.290ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-02-05T23:43:14.334ZINFOtfportd: connected to dpd running softnpu-9f6b8f71
92026-02-05T23:43:14.334ZINFOtfportd: running in sync-only mode
102026-02-05T23:43:14.336ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-02-05T23:43:14.336ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-02-05T23:43:14.337ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 385d555a-b466-4545-bb71-8debbb978f8d, ), sled_id: Some( 6eb8fc83-ced2-4d0c-8566-d24eaab66cb8, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-02-05T23:43:14.337ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-02-05T23:43:14.342ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::e8a8:e2ff:fe41:709a->ea:a8:e2:41:70:9a interface = tfportrear0_0
162026-02-05T23:43:14.346ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-02-05T23:43:14.351ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
182026-02-05T23:43:14.356ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Feb 5 23:43:14 Rereading configuration. ]
20 [ Feb 5 23:43:14 Executing refresh method (:kill). ]
21 [ Feb 5 23:43:14 Stopping because all processes in service exited. ]
22 [ Feb 5 23:43:14 Executing stop method (:kill). ]
23 [ Feb 5 23:43:14 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Feb 5 23:43:14 Method "start" exited with status 0. ]
252026-02-05T23:43:14.883ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-02-05T23:43:14.928ZINFOtfportd: connected to dpd running softnpu-9f6b8f71
272026-02-05T23:43:14.928ZINFOtfportd: running in sync-only mode
282026-02-05T23:43:14.928ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-02-05T23:43:14.930ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-02-05T23:43:14.930ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 385d555a-b466-4545-bb71-8debbb978f8d, sled_id: 6eb8fc83-ced2-4d0c-8566-d24eaab66cb8, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, lot: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 00c6ae36-7824-4e94-afa4-cad1a06a0ef0, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-02-05T23:43:14.930ZDEBGtfportd: created producer ID producer_id = 562307ef-0290-46dd-91b3-9574b14bedc5 unit = oximeter
322026-02-05T23:43:14.931ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 385d555a-b466-4545-bb71-8debbb978f8d, ), sled_id: Some( 6eb8fc83-ced2-4d0c-8566-d24eaab66cb8, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
332026-02-05T23:43:14.941ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-02-05T23:43:14.942ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::fcfc:39ff:fe20:7636->fe:fc:39:20:76:36 interface = tfportrear0_0
352026-02-05T23:43:14.942ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::e8a8:e2ff:fe41:709a->ea:a8:e2:41:70:9a interface = tfportrear0_0
362026-02-05T23:43:14.944ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-02-05T23:43:14.944ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:48253 method = GET path = /{producer_id} unit = producer-server versions = all
382026-02-05T23:43:14.945ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:48253 unit = producer-server
392026-02-05T23:43:14.945ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:48253 unit = producer-server
402026-02-05T23:43:14.945ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:48253 unit = producer-server
412026-02-05T23:43:14.945ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-02-05T23:43:14.946ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-02-05T23:43:14.946ZDEBGtfportd: starting producer registration task unit = producer-server
442026-02-05T23:43:14.946ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:48253 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 562307ef-0290-46dd-91b3-9574b14bedc5 unit = producer-server
452026-02-05T23:43:14.946ZINFOtfportd: created oximeter producer server unit = oximeter
462026-02-05T23:43:14.946ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-02-05T23:43:14.947ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
482026-02-05T23:43:14.948ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-02-05T23:43:14.948ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-02-05T23:42:58.321283212Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502026-02-05T23:43:14.948ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 15772519133576277109 unit = oximeter
512026-02-05T23:43:14.948ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522026-02-05T23:43:14.950ZWARNtfportd: tfportqsfp0_0 has no addrs
532026-02-05T23:43:15.706ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
542026-02-05T23:43:15.710ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
552026-02-05T23:43:15.714ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
562026-02-05T23:43:15.714ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
572026-02-05T23:43:15.715ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-02-05T23:42:58.321143051Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
582026-02-05T23:43:15.715ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 10358392019170590212 unit = oximeter
592026-02-05T23:43:15.715ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
602026-02-05T23:43:15.738ZINFOtfportd: added new ARP entry entry = 10.151.1.174->82:8e:59:6e:45:05 interface = tfportqsfp0_0
612026-02-05T23:43:16.738ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-02-05T23:43:16.743ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
632026-02-05T23:43:16.747ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
642026-02-05T23:43:17.732ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-02-05T23:43:17.736ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
662026-02-05T23:43:17.739ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
672026-02-05T23:43:19.030ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-02-05T23:43:19.035ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
692026-02-05T23:43:19.038ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
702026-02-05T23:43:19.949ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 5.002750622s error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
712026-02-05T23:43:20.279ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("15344921-d32c-4344-9573-7d3c2c7edf22.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("54662b08-fe60-45d6-9ce1-e72c3ee52250.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 649, tv_nsec: 58778084 } }) unit = producer-server
722026-02-05T23:43:20.280ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
732026-02-05T23:43:20.318ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
742026-02-05T23:43:20.318ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
752026-02-05T23:43:20.322ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
762026-02-05T23:43:20.325ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
772026-02-05T23:43:20.885ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
782026-02-05T23:43:20.890ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
792026-02-05T23:43:20.894ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
802026-02-05T23:43:22.168ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
812026-02-05T23:43:22.173ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
822026-02-05T23:43:22.175ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
832026-02-05T23:43:22.988ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
842026-02-05T23:43:22.993ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
852026-02-05T23:43:23.001ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
862026-02-05T23:43:23.555ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
872026-02-05T23:43:23.560ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
882026-02-05T23:43:23.562ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
892026-02-05T23:43:24.130ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902026-02-05T23:43:24.135ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
912026-02-05T23:43:24.137ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
922026-02-05T23:43:24.952ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15772519133576277109 n_samples = 6 unit = oximeter
932026-02-05T23:43:25.092ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942026-02-05T23:43:25.096ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
952026-02-05T23:43:25.099ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
962026-02-05T23:43:25.717ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10358392019170590212 n_samples = 6 unit = oximeter
972026-02-05T23:43:26.182ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
982026-02-05T23:43:26.190ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
992026-02-05T23:43:26.192ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1002026-02-05T23:43:27.186ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1012026-02-05T23:43:27.190ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1022026-02-05T23:43:27.193ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1032026-02-05T23:43:28.387ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1042026-02-05T23:43:28.391ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1052026-02-05T23:43:28.393ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1062026-02-05T23:43:29.269ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1072026-02-05T23:43:29.278ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1082026-02-05T23:43:29.280ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1092026-02-05T23:43:30.186ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1102026-02-05T23:43:30.190ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1112026-02-05T23:43:30.193ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1122026-02-05T23:43:31.097ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1132026-02-05T23:43:31.102ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1142026-02-05T23:43:31.105ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1152026-02-05T23:43:32.008ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1162026-02-05T23:43:32.013ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1172026-02-05T23:43:32.016ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1182026-02-05T23:43:33.126ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1192026-02-05T23:43:33.131ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1202026-02-05T23:43:33.134ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1212026-02-05T23:43:33.725ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1222026-02-05T23:43:33.731ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1232026-02-05T23:43:33.734ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1242026-02-05T23:43:34.277ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1252026-02-05T23:43:34.281ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1262026-02-05T23:43:34.283ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1272026-02-05T23:43:34.955ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15772519133576277109 n_samples = 6 unit = oximeter
1282026-02-05T23:43:35.220ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1292026-02-05T23:43:35.225ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1302026-02-05T23:43:35.228ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1312026-02-05T23:43:35.319ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1322026-02-05T23:43:35.320ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 20.373593221s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1332026-02-05T23:43:36.306ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10358392019170590212 n_samples = 6 unit = oximeter
1342026-02-05T23:43:36.311ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("15344921-d32c-4344-9573-7d3c2c7edf22.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("54662b08-fe60-45d6-9ce1-e72c3ee52250.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 664, tv_nsec: 503497790 } }) unit = producer-server
1352026-02-05T23:43:36.312ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1362026-02-05T23:43:36.344ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1372026-02-05T23:43:37.285ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1382026-02-05T23:43:37.291ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1392026-02-05T23:43:37.294ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1402026-02-05T23:43:38.061ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1412026-02-05T23:43:38.067ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1422026-02-05T23:43:38.070ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1432026-02-05T23:43:39.012ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1442026-02-05T23:43:39.019ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1452026-02-05T23:43:39.031ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1462026-02-05T23:43:39.758ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1472026-02-05T23:43:39.766ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1482026-02-05T23:43:39.770ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1492026-02-05T23:43:40.456ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1502026-02-05T23:43:40.471ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1512026-02-05T23:43:40.490ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1522026-02-05T23:43:41.468ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1532026-02-05T23:43:41.475ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1542026-02-05T23:43:41.480ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1552026-02-05T23:43:42.236ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1562026-02-05T23:43:42.241ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1572026-02-05T23:43:42.245ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1582026-02-05T23:43:42.931ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1592026-02-05T23:43:42.936ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1602026-02-05T23:43:42.938ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1612026-02-05T23:43:44.235ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1622026-02-05T23:43:44.241ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1632026-02-05T23:43:44.245ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1642026-02-05T23:43:44.908ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1652026-02-05T23:43:44.914ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1662026-02-05T23:43:44.918ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1672026-02-05T23:43:45.546ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15772519133576277109 n_samples = 6 unit = oximeter
1682026-02-05T23:43:46.311ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10358392019170590212 n_samples = 6 unit = oximeter
1692026-02-05T23:43:46.357ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1702026-02-05T23:43:46.367ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1712026-02-05T23:43:46.370ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1722026-02-05T23:43:47.838ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1732026-02-05T23:43:47.845ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1742026-02-05T23:43:47.848ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1752026-02-05T23:43:49.202ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1762026-02-05T23:43:49.208ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1772026-02-05T23:43:49.212ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1782026-02-05T23:43:50.137ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1792026-02-05T23:43:50.156ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1802026-02-05T23:43:54.175ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1812026-02-05T23:43:54.179ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1822026-02-05T23:43:54.188ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 38.651282106s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1832026-02-05T23:43:54.914ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("15344921-d32c-4344-9573-7d3c2c7edf22.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("54662b08-fe60-45d6-9ce1-e72c3ee52250.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 683, tv_nsec: 102640303 } }) unit = producer-server
1842026-02-05T23:43:54.915ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1852026-02-05T23:43:54.967ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1862026-02-05T23:43:55.424ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1872026-02-05T23:43:55.432ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1882026-02-05T23:43:55.436ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1892026-02-05T23:43:55.559ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15772519133576277109 n_samples = 6 unit = oximeter
1902026-02-05T23:43:56.315ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10358392019170590212 n_samples = 6 unit = oximeter
1912026-02-05T23:43:56.832ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1922026-02-05T23:43:56.837ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1932026-02-05T23:43:56.840ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1942026-02-05T23:43:57.530ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1952026-02-05T23:43:57.535ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1962026-02-05T23:43:57.538ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1972026-02-05T23:43:58.997ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1982026-02-05T23:43:59.001ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
1992026-02-05T23:43:59.003ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2002026-02-05T23:44:00.400ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2012026-02-05T23:44:00.406ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2022026-02-05T23:44:00.409ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2032026-02-05T23:44:01.093ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2042026-02-05T23:44:01.099ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2052026-02-05T23:44:01.102ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2062026-02-05T23:44:02.074ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2072026-02-05T23:44:02.079ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2082026-02-05T23:44:02.083ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2092026-02-05T23:44:02.721ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2102026-02-05T23:44:02.727ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2112026-02-05T23:44:02.730ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2122026-02-05T23:44:03.497ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2132026-02-05T23:44:03.503ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2142026-02-05T23:44:03.506ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2152026-02-05T23:44:04.054ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2162026-02-05T23:44:04.059ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2172026-02-05T23:44:04.062ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2182026-02-05T23:44:04.855ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2192026-02-05T23:44:04.860ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2202026-02-05T23:44:04.863ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2212026-02-05T23:44:05.561ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15772519133576277109 n_samples = 6 unit = oximeter
2222026-02-05T23:44:06.208ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2232026-02-05T23:44:06.215ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2242026-02-05T23:44:06.218ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2252026-02-05T23:44:06.318ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10358392019170590212 n_samples = 6 unit = oximeter
2262026-02-05T23:44:07.398ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2272026-02-05T23:44:07.404ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2282026-02-05T23:44:07.406ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2292026-02-05T23:44:08.774ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2302026-02-05T23:44:08.779ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2312026-02-05T23:44:08.782ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2322026-02-05T23:44:09.695ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2332026-02-05T23:44:09.699ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2342026-02-05T23:44:09.701ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2352026-02-05T23:44:09.968ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2362026-02-05T23:44:09.970ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 54.433390387s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2372026-02-05T23:44:10.792ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2382026-02-05T23:44:10.797ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2392026-02-05T23:44:10.799ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2402026-02-05T23:44:11.766ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2412026-02-05T23:44:11.772ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2422026-02-05T23:44:11.775ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2432026-02-05T23:44:11.813ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("15344921-d32c-4344-9573-7d3c2c7edf22.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("54662b08-fe60-45d6-9ce1-e72c3ee52250.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 700, tv_nsec: 2355547 } }) unit = producer-server
2442026-02-05T23:44:11.813ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2452026-02-05T23:44:11.847ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2462026-02-05T23:44:12.387ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2472026-02-05T23:44:12.392ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2482026-02-05T23:44:12.394ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2492026-02-05T23:44:13.780ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2502026-02-05T23:44:13.786ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2512026-02-05T23:44:13.789ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2522026-02-05T23:44:14.684ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2532026-02-05T23:44:14.689ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2542026-02-05T23:44:14.692ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2552026-02-05T23:44:15.563ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15772519133576277109 n_samples = 6 unit = oximeter
2562026-02-05T23:44:16.074ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2572026-02-05T23:44:16.079ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2582026-02-05T23:44:16.083ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2592026-02-05T23:44:16.319ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10358392019170590212 n_samples = 6 unit = oximeter
2602026-02-05T23:44:17.096ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2612026-02-05T23:44:17.103ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2622026-02-05T23:44:17.106ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2632026-02-05T23:44:17.678ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2642026-02-05T23:44:17.684ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2652026-02-05T23:44:17.688ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2662026-02-05T23:44:19.031ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2672026-02-05T23:44:19.037ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2682026-02-05T23:44:19.040ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2692026-02-05T23:44:20.237ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2702026-02-05T23:44:20.242ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2712026-02-05T23:44:20.245ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2722026-02-05T23:44:21.066ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2732026-02-05T23:44:21.072ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2742026-02-05T23:44:21.075ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2752026-02-05T23:44:22.380ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2762026-02-05T23:44:22.390ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2772026-02-05T23:44:22.394ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2782026-02-05T23:44:23.130ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2792026-02-05T23:44:23.153ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2802026-02-05T23:44:23.160ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2812026-02-05T23:44:30.895ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2822026-02-05T23:44:31.006ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 75.468440408s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2832026-02-05T23:44:31.240ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15772519133576277109 n_samples = 6 unit = oximeter
2842026-02-05T23:44:31.244ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10358392019170590212 n_samples = 6 unit = oximeter
2852026-02-05T23:44:31.296ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2862026-02-05T23:44:31.475ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2872026-02-05T23:44:31.551ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2882026-02-05T23:44:32.501ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2892026-02-05T23:44:32.511ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2902026-02-05T23:44:32.517ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2912026-02-05T23:44:33.059ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("15344921-d32c-4344-9573-7d3c2c7edf22.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("54662b08-fe60-45d6-9ce1-e72c3ee52250.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 721, tv_nsec: 247393192 } }) unit = producer-server
2922026-02-05T23:44:33.063ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2932026-02-05T23:44:33.144ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2942026-02-05T23:44:33.661ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2952026-02-05T23:44:33.677ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2962026-02-05T23:44:33.682ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2972026-02-05T23:44:35.116ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2982026-02-05T23:44:35.148ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
2992026-02-05T23:44:35.156ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3002026-02-05T23:44:35.968ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3012026-02-05T23:44:35.975ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3022026-02-05T23:44:35.979ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3032026-02-05T23:44:36.810ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3042026-02-05T23:44:36.815ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3052026-02-05T23:44:36.819ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3062026-02-05T23:44:38.135ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3072026-02-05T23:44:38.141ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3082026-02-05T23:44:38.146ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3092026-02-05T23:44:39.374ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3102026-02-05T23:44:39.379ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3112026-02-05T23:44:39.382ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3122026-02-05T23:44:40.169ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3132026-02-05T23:44:40.174ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3142026-02-05T23:44:40.178ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3152026-02-05T23:44:41.312ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15772519133576277109 n_samples = 6 unit = oximeter
3162026-02-05T23:44:41.314ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10358392019170590212 n_samples = 6 unit = oximeter
3172026-02-05T23:44:41.470ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3182026-02-05T23:44:41.476ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3192026-02-05T23:44:41.482ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3202026-02-05T23:44:42.273ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3212026-02-05T23:44:42.279ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3222026-02-05T23:44:42.283ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3232026-02-05T23:44:43.572ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3242026-02-05T23:44:43.577ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3252026-02-05T23:44:43.580ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3262026-02-05T23:44:44.517ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3272026-02-05T23:44:44.522ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3282026-02-05T23:44:44.525ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3292026-02-05T23:44:46.034ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3302026-02-05T23:44:46.040ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3312026-02-05T23:44:46.044ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3322026-02-05T23:44:46.991ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3332026-02-05T23:44:46.998ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3342026-02-05T23:44:47.002ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3352026-02-05T23:44:48.147ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3362026-02-05T23:44:48.151ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 92.613454002s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3372026-02-05T23:44:48.236ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382026-02-05T23:44:48.241ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3392026-02-05T23:44:48.244ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3402026-02-05T23:44:49.705ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3412026-02-05T23:44:49.710ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3422026-02-05T23:44:49.714ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3432026-02-05T23:44:50.332ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3442026-02-05T23:44:50.338ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3452026-02-05T23:44:50.342ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3462026-02-05T23:44:51.316ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15772519133576277109 n_samples = 6 unit = oximeter
3472026-02-05T23:44:51.317ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10358392019170590212 n_samples = 6 unit = oximeter
3482026-02-05T23:44:51.689ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3492026-02-05T23:44:51.695ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3502026-02-05T23:44:51.698ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3512026-02-05T23:44:53.113ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3522026-02-05T23:44:53.117ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3532026-02-05T23:44:53.120ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3542026-02-05T23:44:53.767ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3552026-02-05T23:44:53.773ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3562026-02-05T23:44:53.776ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3572026-02-05T23:44:55.056ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3582026-02-05T23:44:55.064ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3592026-02-05T23:44:55.068ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3602026-02-05T23:44:56.305ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3612026-02-05T23:44:56.311ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3622026-02-05T23:44:56.314ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3632026-02-05T23:44:57.470ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3642026-02-05T23:44:57.476ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3652026-02-05T23:44:57.479ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3662026-02-05T23:44:57.681ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("15344921-d32c-4344-9573-7d3c2c7edf22.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("54662b08-fe60-45d6-9ce1-e72c3ee52250.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e1eee7a7-81d9-4ae6-a345-867b61a44c27.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 745, tv_nsec: 866607075 } }) unit = producer-server
3672026-02-05T23:44:57.682ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
3682026-02-05T23:44:57.820ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3692026-02-05T23:44:58.412ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3702026-02-05T23:44:58.419ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3712026-02-05T23:44:58.423ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3722026-02-05T23:44:58.957ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3732026-02-05T23:44:58.966ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3742026-02-05T23:44:58.971ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3752026-02-05T23:45:00.296ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3762026-02-05T23:45:00.304ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3772026-02-05T23:45:00.308ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3782026-02-05T23:45:01.296ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3792026-02-05T23:45:01.305ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3802026-02-05T23:45:01.310ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3812026-02-05T23:45:01.320ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 15772519133576277109 n_samples = 6 unit = oximeter
3822026-02-05T23:45:01.322ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 10358392019170590212 n_samples = 6 unit = oximeter
3832026-02-05T23:45:01.892ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3842026-02-05T23:45:01.902ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e8a8:e2ff:fe41:709a]
3852026-02-05T23:45:01.907ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]