Max level shown:
1 [ Feb 23 16:27:58 Enabled. ]
2 [ Feb 23 16:27:58 Rereading configuration. ]
3 [ Feb 23 16:27:59 Rereading configuration. ]
4 [ Feb 23 16:28:10 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Feb 23 16:28:10 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-02-23T16:28:10.239ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-02-23T16:28:10.283ZINFOtfportd: connected to dpd running softnpu-37992295
92026-02-23T16:28:10.283ZINFOtfportd: running in sync-only mode
102026-02-23T16:28:10.285ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-02-23T16:28:10.286ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-02-23T16:28:10.287ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( e95a8812-f23d-4e3c-8ac7-7c52450463d1, ), sled_id: Some( 5bf4b99e-071e-4402-bb55-2790f4933e69, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-02-23T16:28:10.287ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-02-23T16:28:10.292ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::e88f:b4ff:fe4a:7149->ea:8f:b4:4a:71:49 interface = tfportrear0_0
162026-02-23T16:28:10.293ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-02-23T16:28:10.298ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
182026-02-23T16:28:10.303ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Feb 23 16:28:10 Rereading configuration. ]
20 [ Feb 23 16:28:10 Executing refresh method (:kill). ]
21 [ Feb 23 16:28:10 Stopping because all processes in service exited. ]
22 [ Feb 23 16:28:10 Executing stop method (:kill). ]
23 [ Feb 23 16:28:10 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Feb 23 16:28:10 Method "start" exited with status 0. ]
252026-02-23T16:28:10.956ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-02-23T16:28:11.007ZINFOtfportd: connected to dpd running softnpu-37992295
272026-02-23T16:28:11.007ZINFOtfportd: running in sync-only mode
282026-02-23T16:28:11.007ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-02-23T16:28:11.009ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-02-23T16:28:11.009ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: e95a8812-f23d-4e3c-8ac7-7c52450463d1, sled_id: 5bf4b99e-071e-4402-bb55-2790f4933e69, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, lot: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: ffea4e13-f5e9-4241-bab9-529934710d0c, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-02-23T16:28:11.009ZDEBGtfportd: created producer ID producer_id = 4d447f9f-00ca-487f-8b2b-095b85e650b4 unit = oximeter
322026-02-23T16:28:11.011ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( e95a8812-f23d-4e3c-8ac7-7c52450463d1, ), sled_id: Some( 5bf4b99e-071e-4402-bb55-2790f4933e69, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
332026-02-23T16:28:11.019ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-02-23T16:28:11.021ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::bcaf:45ff:fe49:618f->be:af:45:49:61:8f interface = tfportrear0_0
352026-02-23T16:28:11.021ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::e88f:b4ff:fe4a:7149->ea:8f:b4:4a:71:49 interface = tfportrear0_0
362026-02-23T16:28:11.023ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-02-23T16:28:11.024ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
382026-02-23T16:28:11.024ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:39869 method = GET path = /{producer_id} unit = producer-server versions = all
392026-02-23T16:28:11.024ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:39869 unit = producer-server
402026-02-23T16:28:11.025ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:39869 unit = producer-server
412026-02-23T16:28:11.025ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:39869 unit = producer-server
422026-02-23T16:28:11.025ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
432026-02-23T16:28:11.025ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
442026-02-23T16:28:11.026ZDEBGtfportd: starting producer registration task unit = producer-server
452026-02-23T16:28:11.026ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:39869 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 4d447f9f-00ca-487f-8b2b-095b85e650b4 unit = producer-server
462026-02-23T16:28:11.026ZINFOtfportd: created oximeter producer server unit = oximeter
472026-02-23T16:28:11.026ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
482026-02-23T16:28:11.026ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-02-23T16:28:11.027ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-02-23T16:27:54.173223343Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502026-02-23T16:28:11.027ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 9372155628124625121 unit = oximeter
512026-02-23T16:28:11.027ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522026-02-23T16:28:11.028ZWARNtfportd: tfportqsfp0_0 has no addrs
532026-02-23T16:28:12.008ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
542026-02-23T16:28:12.014ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
552026-02-23T16:28:12.017ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
562026-02-23T16:28:12.018ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
572026-02-23T16:28:12.018ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-02-23T16:27:54.173065071Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
582026-02-23T16:28:12.019ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 16008137382151199390 unit = oximeter
592026-02-23T16:28:12.019ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
602026-02-23T16:28:12.208ZINFOtfportd: added new ARP entry entry = 10.151.1.174->da:8a:6f:6b:04:b0 interface = tfportqsfp0_0
612026-02-23T16:28:13.232ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-02-23T16:28:13.236ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
632026-02-23T16:28:13.238ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
642026-02-23T16:28:14.235ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-02-23T16:28:14.241ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
662026-02-23T16:28:14.243ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
672026-02-23T16:28:15.239ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-02-23T16:28:15.244ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
692026-02-23T16:28:15.246ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
702026-02-23T16:28:16.021ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-02-23T16:28:16.028ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
722026-02-23T16:28:16.028ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("22821aba-0e42-4251-bba1-cfd3795ce2e0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("898d20d1-89b6-49fd-bdf8-f7f44c462267.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 1025, tv_nsec: 983687975 } }) unit = producer-server
732026-02-23T16:28:16.029ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
742026-02-23T16:28:16.064ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
752026-02-23T16:28:16.064ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
762026-02-23T16:28:17.207ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-02-23T16:28:17.211ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
782026-02-23T16:28:17.214ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
792026-02-23T16:28:18.221ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-02-23T16:28:18.225ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
812026-02-23T16:28:18.228ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
822026-02-23T16:28:18.898ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-02-23T16:28:18.902ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
842026-02-23T16:28:18.905ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
852026-02-23T16:28:19.847ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862026-02-23T16:28:19.851ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
872026-02-23T16:28:19.854ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
882026-02-23T16:28:20.418ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
892026-02-23T16:28:20.422ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
902026-02-23T16:28:20.425ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
912026-02-23T16:28:21.033ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9372155628124625121 n_samples = 6 unit = oximeter
922026-02-23T16:28:21.566ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
932026-02-23T16:28:21.570ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
942026-02-23T16:28:21.573ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
952026-02-23T16:28:22.020ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16008137382151199390 n_samples = 6 unit = oximeter
962026-02-23T16:28:22.856ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-02-23T16:28:22.861ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
982026-02-23T16:28:22.864ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
992026-02-23T16:28:23.951ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002026-02-23T16:28:23.955ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1012026-02-23T16:28:23.958ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1022026-02-23T16:28:24.035ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-02-23T16:28:24.039ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1042026-02-23T16:28:24.042ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1052026-02-23T16:28:25.194ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-02-23T16:28:25.199ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1072026-02-23T16:28:25.202ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1082026-02-23T16:28:26.250ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-02-23T16:28:26.255ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1102026-02-23T16:28:26.258ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1112026-02-23T16:28:26.837ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-02-23T16:28:26.844ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1132026-02-23T16:28:26.847ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1142026-02-23T16:28:28.255ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-02-23T16:28:28.258ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1162026-02-23T16:28:28.261ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1172026-02-23T16:28:29.013ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1182026-02-23T16:28:29.018ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1192026-02-23T16:28:29.020ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1202026-02-23T16:28:30.519ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1212026-02-23T16:28:30.524ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1222026-02-23T16:28:30.527ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1232026-02-23T16:28:30.654ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9372155628124625121 n_samples = 6 unit = oximeter
1242026-02-23T16:28:30.683ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1252026-02-23T16:28:30.684ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 20.039904904s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1262026-02-23T16:28:30.973ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("22821aba-0e42-4251-bba1-cfd3795ce2e0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("898d20d1-89b6-49fd-bdf8-f7f44c462267.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 1041, tv_nsec: 310830350 } }) unit = producer-server
1272026-02-23T16:28:30.973ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1282026-02-23T16:28:31.006ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1292026-02-23T16:28:31.639ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16008137382151199390 n_samples = 6 unit = oximeter
1302026-02-23T16:28:32.017ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312026-02-23T16:28:32.022ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1322026-02-23T16:28:32.025ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1332026-02-23T16:28:33.040ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342026-02-23T16:28:33.044ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1352026-02-23T16:28:33.047ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1362026-02-23T16:28:33.714ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-02-23T16:28:33.719ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1382026-02-23T16:28:33.721ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1392026-02-23T16:28:35.014ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-02-23T16:28:35.019ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1412026-02-23T16:28:35.022ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1422026-02-23T16:28:35.818ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-02-23T16:28:35.822ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1442026-02-23T16:28:35.825ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1452026-02-23T16:28:37.041ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-02-23T16:28:37.045ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1472026-02-23T16:28:37.048ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1482026-02-23T16:28:38.242ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-02-23T16:28:38.246ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1502026-02-23T16:28:38.249ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1512026-02-23T16:28:38.814ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-02-23T16:28:38.818ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1532026-02-23T16:28:38.821ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1542026-02-23T16:28:40.230ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1552026-02-23T16:28:40.235ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1562026-02-23T16:28:40.238ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1572026-02-23T16:28:40.629ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9372155628124625121 n_samples = 6 unit = oximeter
1582026-02-23T16:28:40.810ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1592026-02-23T16:28:40.815ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1602026-02-23T16:28:40.818ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1612026-02-23T16:28:41.613ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16008137382151199390 n_samples = 6 unit = oximeter
1622026-02-23T16:28:41.959ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1632026-02-23T16:28:41.964ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1642026-02-23T16:28:41.967ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1652026-02-23T16:28:43.049ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1662026-02-23T16:28:43.054ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1672026-02-23T16:28:43.058ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1682026-02-23T16:28:44.106ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1692026-02-23T16:28:44.110ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1702026-02-23T16:28:44.114ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1712026-02-23T16:28:45.159ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-02-23T16:28:45.164ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1732026-02-23T16:28:45.166ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1742026-02-23T16:28:45.966ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1752026-02-23T16:28:45.967ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 35.364261812s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1762026-02-23T16:28:46.370ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1772026-02-23T16:28:46.375ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1782026-02-23T16:28:46.378ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1792026-02-23T16:28:46.595ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("22821aba-0e42-4251-bba1-cfd3795ce2e0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("898d20d1-89b6-49fd-bdf8-f7f44c462267.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 1056, tv_nsec: 975038157 } }) unit = producer-server
1802026-02-23T16:28:46.596ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1812026-02-23T16:28:46.627ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1822026-02-23T16:28:47.649ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1832026-02-23T16:28:47.654ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1842026-02-23T16:28:47.656ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1852026-02-23T16:28:48.952ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862026-02-23T16:28:48.956ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1872026-02-23T16:28:48.959ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1882026-02-23T16:28:50.154ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-02-23T16:28:50.159ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1902026-02-23T16:28:50.162ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1912026-02-23T16:28:50.604ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9372155628124625121 n_samples = 6 unit = oximeter
1922026-02-23T16:28:50.831ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1932026-02-23T16:28:50.836ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1942026-02-23T16:28:50.839ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1952026-02-23T16:28:51.587ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16008137382151199390 n_samples = 6 unit = oximeter
1962026-02-23T16:28:51.914ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1972026-02-23T16:28:51.918ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
1982026-02-23T16:28:51.921ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1992026-02-23T16:28:52.876ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-02-23T16:28:52.880ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2012026-02-23T16:28:52.883ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2022026-02-23T16:28:54.219ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032026-02-23T16:28:54.223ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2042026-02-23T16:28:54.225ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2052026-02-23T16:28:55.518ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2062026-02-23T16:28:55.526ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2072026-02-23T16:28:55.530ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2082026-02-23T16:28:56.317ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-02-23T16:28:56.328ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2102026-02-23T16:28:56.331ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2112026-02-23T16:28:57.688ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-02-23T16:28:57.694ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2132026-02-23T16:28:57.704ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2142026-02-23T16:28:58.798ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2152026-02-23T16:28:58.804ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2162026-02-23T16:28:58.807ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2172026-02-23T16:28:59.594ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2182026-02-23T16:28:59.606ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2192026-02-23T16:28:59.627ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2202026-02-23T16:29:00.337ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2212026-02-23T16:29:00.344ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2222026-02-23T16:29:00.348ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2232026-02-23T16:29:00.581ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9372155628124625121 n_samples = 6 unit = oximeter
2242026-02-23T16:29:01.563ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16008137382151199390 n_samples = 6 unit = oximeter
2252026-02-23T16:29:01.589ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2262026-02-23T16:29:01.591ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 51.030447078s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2272026-02-23T16:29:01.783ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2282026-02-23T16:29:01.788ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2292026-02-23T16:29:01.792ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2302026-02-23T16:29:02.469ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("22821aba-0e42-4251-bba1-cfd3795ce2e0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("898d20d1-89b6-49fd-bdf8-f7f44c462267.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 1072, tv_nsec: 891928101 } }) unit = producer-server
2312026-02-23T16:29:02.470ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2322026-02-23T16:29:02.521ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2332026-02-23T16:29:03.186ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-02-23T16:29:03.190ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2352026-02-23T16:29:03.194ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2362026-02-23T16:29:04.679ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372026-02-23T16:29:04.684ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2382026-02-23T16:29:04.687ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2392026-02-23T16:29:05.719ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402026-02-23T16:29:05.726ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2412026-02-23T16:29:05.730ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2422026-02-23T16:29:06.502ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2432026-02-23T16:29:06.507ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2442026-02-23T16:29:06.509ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2452026-02-23T16:29:13.810ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462026-02-23T16:29:13.819ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2472026-02-23T16:29:13.835ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9372155628124625121 n_samples = 6 unit = oximeter
2482026-02-23T16:29:13.836ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16008137382151199390 n_samples = 6 unit = oximeter
2492026-02-23T16:29:13.848ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2502026-02-23T16:29:15.049ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2512026-02-23T16:29:15.056ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2522026-02-23T16:29:15.061ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2532026-02-23T16:29:16.342ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2542026-02-23T16:29:16.348ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2552026-02-23T16:29:16.392ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2562026-02-23T16:29:16.462ZINFOtfportd: removed stale NDP entry :fe80::bcaf:45ff:fe49:618f->be:af:45:49:61:8f
2572026-02-23T16:29:17.144ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2582026-02-23T16:29:17.152ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2592026-02-23T16:29:17.156ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2602026-02-23T16:29:17.498ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2612026-02-23T16:29:17.506ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 66.976240137s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2622026-02-23T16:29:18.222ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2632026-02-23T16:29:18.228ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2642026-02-23T16:29:18.232ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2652026-02-23T16:29:18.658ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::bcaf:45ff:fe49:618f->be:af:45:49:61:8f interface = tfportrear0_0
2662026-02-23T16:29:18.805ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2672026-02-23T16:29:18.813ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2682026-02-23T16:29:18.817ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2692026-02-23T16:29:19.858ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("22821aba-0e42-4251-bba1-cfd3795ce2e0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("898d20d1-89b6-49fd-bdf8-f7f44c462267.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 1090, tv_nsec: 306864236 } }) unit = producer-server
2702026-02-23T16:29:19.860ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2712026-02-23T16:29:19.871ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722026-02-23T16:29:19.875ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2732026-02-23T16:29:19.878ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2742026-02-23T16:29:19.901ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2752026-02-23T16:29:20.970ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2762026-02-23T16:29:20.974ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2772026-02-23T16:29:20.977ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2782026-02-23T16:29:21.898ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2792026-02-23T16:29:21.903ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2802026-02-23T16:29:21.906ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2812026-02-23T16:29:22.695ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2822026-02-23T16:29:22.700ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2832026-02-23T16:29:22.702ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2842026-02-23T16:29:23.394ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2852026-02-23T16:29:23.399ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2862026-02-23T16:29:23.402ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2872026-02-23T16:29:23.848ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9372155628124625121 n_samples = 6 unit = oximeter
2882026-02-23T16:29:23.848ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16008137382151199390 n_samples = 6 unit = oximeter
2892026-02-23T16:29:24.830ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2902026-02-23T16:29:24.835ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2912026-02-23T16:29:24.838ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2922026-02-23T16:29:25.937ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2932026-02-23T16:29:25.943ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2942026-02-23T16:29:25.946ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2952026-02-23T16:29:27.310ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2962026-02-23T16:29:27.315ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
2972026-02-23T16:29:27.318ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2982026-02-23T16:29:28.237ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2992026-02-23T16:29:28.242ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3002026-02-23T16:29:28.246ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3012026-02-23T16:29:29.417ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3022026-02-23T16:29:29.422ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3032026-02-23T16:29:29.425ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3042026-02-23T16:29:30.583ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3052026-02-23T16:29:30.589ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3062026-02-23T16:29:30.592ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3072026-02-23T16:29:31.338ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3082026-02-23T16:29:31.343ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3092026-02-23T16:29:31.346ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3102026-02-23T16:29:31.916ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3112026-02-23T16:29:31.922ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3122026-02-23T16:29:31.925ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3132026-02-23T16:29:32.680ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3142026-02-23T16:29:32.685ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3152026-02-23T16:29:32.688ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3162026-02-23T16:29:33.854ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9372155628124625121 n_samples = 6 unit = oximeter
3172026-02-23T16:29:33.855ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16008137382151199390 n_samples = 6 unit = oximeter
3182026-02-23T16:29:34.033ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3192026-02-23T16:29:34.038ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3202026-02-23T16:29:34.040ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3212026-02-23T16:29:34.909ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3222026-02-23T16:29:34.910ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 84.370758232s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3232026-02-23T16:29:35.483ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3242026-02-23T16:29:35.489ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3252026-02-23T16:29:35.492ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3262026-02-23T16:29:36.295ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3272026-02-23T16:29:36.300ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3282026-02-23T16:29:36.303ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3292026-02-23T16:29:37.569ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3302026-02-23T16:29:37.574ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3312026-02-23T16:29:37.577ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3322026-02-23T16:29:38.449ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3332026-02-23T16:29:38.454ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3342026-02-23T16:29:38.457ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3352026-02-23T16:29:38.977ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3362026-02-23T16:29:38.982ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3372026-02-23T16:29:38.985ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3382026-02-23T16:29:39.529ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3392026-02-23T16:29:39.535ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3402026-02-23T16:29:39.538ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3412026-02-23T16:29:39.995ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("22821aba-0e42-4251-bba1-cfd3795ce2e0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("898d20d1-89b6-49fd-bdf8-f7f44c462267.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e3fd166e-83b4-4105-9fe2-3b5421d608da.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 1110, tv_nsec: 437875886 } }) unit = producer-server
3422026-02-23T16:29:39.996ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
3432026-02-23T16:29:40.028ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3442026-02-23T16:29:40.925ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3452026-02-23T16:29:40.931ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3462026-02-23T16:29:40.933ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3472026-02-23T16:29:42.351ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3482026-02-23T16:29:42.356ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3492026-02-23T16:29:42.360ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3502026-02-23T16:29:43.784ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3512026-02-23T16:29:43.790ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3522026-02-23T16:29:43.793ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3532026-02-23T16:29:43.856ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9372155628124625121 n_samples = 6 unit = oximeter
3542026-02-23T16:29:43.869ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16008137382151199390 n_samples = 6 unit = oximeter
3552026-02-23T16:29:45.194ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3562026-02-23T16:29:45.200ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3572026-02-23T16:29:45.203ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3582026-02-23T16:29:46.142ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3592026-02-23T16:29:46.147ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3602026-02-23T16:29:46.150ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3612026-02-23T16:29:47.220ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3622026-02-23T16:29:47.226ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3632026-02-23T16:29:47.229ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3642026-02-23T16:29:48.195ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3652026-02-23T16:29:48.200ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3662026-02-23T16:29:48.203ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3672026-02-23T16:29:49.206ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3682026-02-23T16:29:49.211ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3692026-02-23T16:29:49.215ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3702026-02-23T16:29:50.281ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3712026-02-23T16:29:50.287ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3722026-02-23T16:29:50.290ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3732026-02-23T16:29:51.367ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3742026-02-23T16:29:51.373ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3752026-02-23T16:29:51.376ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3762026-02-23T16:29:52.437ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3772026-02-23T16:29:52.443ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3782026-02-23T16:29:52.446ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3792026-02-23T16:29:53.760ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3802026-02-23T16:29:53.766ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3812026-02-23T16:29:53.769ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3822026-02-23T16:29:53.870ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 9372155628124625121 n_samples = 6 unit = oximeter
3832026-02-23T16:29:53.871ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 16008137382151199390 n_samples = 6 unit = oximeter
3842026-02-23T16:29:54.884ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3852026-02-23T16:29:54.889ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3862026-02-23T16:29:54.892ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3872026-02-23T16:29:55.029ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3882026-02-23T16:29:55.030ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 104.491155023s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3892026-02-23T16:29:55.654ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3902026-02-23T16:29:55.660ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3912026-02-23T16:29:55.663ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3922026-02-23T16:29:56.504ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3932026-02-23T16:29:56.509ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3942026-02-23T16:29:56.516ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3952026-02-23T16:29:57.681ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3962026-02-23T16:29:57.687ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
3972026-02-23T16:29:57.690ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3982026-02-23T16:29:58.213ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3992026-02-23T16:29:58.218ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
4002026-02-23T16:29:58.221ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4012026-02-23T16:29:58.885ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4022026-02-23T16:29:58.891ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
4032026-02-23T16:29:58.894ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4042026-02-23T16:29:59.617ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4052026-02-23T16:29:59.623ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::e88f:b4ff:fe4a:7149]
4062026-02-23T16:29:59.626ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]