Max level shown:
1 [ Jan 5 22:27:14 Enabled. ]
2 [ Jan 5 22:27:14 Rereading configuration. ]
3 [ Jan 5 22:27:15 Rereading configuration. ]
4 [ Jan 5 22:27:26 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jan 5 22:27:26 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-01-05T22:27:26.185ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-01-05T22:27:26.227ZINFOtfportd: connected to dpd running softnpu-c9bf538d
92026-01-05T22:27:26.227ZINFOtfportd: running in sync-only mode
102026-01-05T22:27:26.228ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-01-05T22:27:26.230ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-01-05T22:27:26.232ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 111726fb-5486-4b99-9215-e3139cbbb985, ), sled_id: Some( 2638912c-5b83-48b2-9e20-3a93cc7b05dd, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
142026-01-05T22:27:26.232ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-01-05T22:27:26.237ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::24f8:78ff:fe78:55d4->26:f8:78:78:55:d4 interface = tfportrear0_0
162026-01-05T22:27:26.240ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-01-05T22:27:26.247ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
182026-01-05T22:27:26.254ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Jan 5 22:27:26 Rereading configuration. ]
20 [ Jan 5 22:27:26 Executing refresh method (:kill). ]
21 [ Jan 5 22:27:26 Stopping because all processes in service exited. ]
22 [ Jan 5 22:27:26 Executing stop method (:kill). ]
23 [ Jan 5 22:27:26 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Jan 5 22:27:26 Method "start" exited with status 0. ]
252026-01-05T22:27:26.869ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-01-05T22:27:26.909ZINFOtfportd: connected to dpd running softnpu-c9bf538d
272026-01-05T22:27:26.909ZINFOtfportd: running in sync-only mode
282026-01-05T22:27:26.909ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-01-05T22:27:26.911ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-01-05T22:27:26.911ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 111726fb-5486-4b99-9215-e3139cbbb985, sled_id: 2638912c-5b83-48b2-9e20-3a93cc7b05dd, model: "i86pc", revision: 0, serial: "buskin" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, lot: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 264a8eb8-4b22-4360-9a62-66551e759551, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-01-05T22:27:26.911ZDEBGtfportd: created producer ID producer_id = 1165396f-4427-404b-9657-bfd54eb52909 unit = oximeter
322026-01-05T22:27:26.912ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 111726fb-5486-4b99-9215-e3139cbbb985, ), sled_id: Some( 2638912c-5b83-48b2-9e20-3a93cc7b05dd, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "buskin", ), }
332026-01-05T22:27:26.919ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-01-05T22:27:26.921ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::a2:2ff:fe8f:3dc1->02:a2:02:8f:3d:c1 interface = tfportrear0_0
352026-01-05T22:27:26.921ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::24f8:78ff:fe78:55d4->26:f8:78:78:55:d4 interface = tfportrear0_0
362026-01-05T22:27:26.922ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-01-05T22:27:26.923ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:45850 method = GET path = /{producer_id} unit = producer-server versions = all
382026-01-05T22:27:26.923ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:45850 unit = producer-server
392026-01-05T22:27:26.924ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:45850 unit = producer-server
402026-01-05T22:27:26.924ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:45850 unit = producer-server
412026-01-05T22:27:26.924ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-01-05T22:27:26.924ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-01-05T22:27:26.924ZDEBGtfportd: starting producer registration task unit = producer-server
442026-01-05T22:27:26.924ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:45850 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 1165396f-4427-404b-9657-bfd54eb52909 unit = producer-server
452026-01-05T22:27:26.924ZINFOtfportd: created oximeter producer server unit = oximeter
462026-01-05T22:27:26.924ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-01-05T22:27:26.924ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
482026-01-05T22:27:26.925ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-01-05T22:27:26.925ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-01-05T22:27:10.229526858Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502026-01-05T22:27:26.925ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 7736234662763567125 unit = oximeter
512026-01-05T22:27:26.925ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522026-01-05T22:27:26.928ZWARNtfportd: tfportqsfp0_0 has no addrs
532026-01-05T22:27:28.058ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
542026-01-05T22:27:28.063ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
552026-01-05T22:27:28.067ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
562026-01-05T22:27:28.067ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
572026-01-05T22:27:28.069ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-01-05T22:27:10.229393943Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
582026-01-05T22:27:28.069ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 5090599079127267502 unit = oximeter
592026-01-05T22:27:28.069ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
602026-01-05T22:27:28.220ZINFOtfportd: added new ARP entry entry = 10.151.1.174->fa:a6:ae:2d:1f:f6 interface = tfportqsfp0_0
612026-01-05T22:27:29.576ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-01-05T22:27:29.581ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
632026-01-05T22:27:29.583ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
642026-01-05T22:27:30.395ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-01-05T22:27:30.400ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
662026-01-05T22:27:30.403ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
672026-01-05T22:27:31.664ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-01-05T22:27:31.668ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
692026-01-05T22:27:31.670ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
702026-01-05T22:27:31.927ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("77861647-cebf-4e85-bdde-48b1a190f0ce.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("98249b9a-2f6e-46a4-be1e-673a1bba11fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 664, tv_nsec: 692489391 } }) unit = producer-server
712026-01-05T22:27:31.928ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
722026-01-05T22:27:31.962ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
732026-01-05T22:27:32.778ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-01-05T22:27:32.782ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
752026-01-05T22:27:32.785ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
762026-01-05T22:27:33.612ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-01-05T22:27:33.617ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
782026-01-05T22:27:33.620ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
792026-01-05T22:27:35.082ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-01-05T22:27:35.086ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
812026-01-05T22:27:35.089ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
822026-01-05T22:27:36.202ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-01-05T22:27:36.207ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
842026-01-05T22:27:36.210ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
852026-01-05T22:27:36.930ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
862026-01-05T22:27:37.445ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
872026-01-05T22:27:37.450ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
882026-01-05T22:27:37.453ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
892026-01-05T22:27:38.072ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
902026-01-05T22:27:38.370ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
912026-01-05T22:27:38.375ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
922026-01-05T22:27:38.378ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
932026-01-05T22:27:39.061ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942026-01-05T22:27:39.066ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
952026-01-05T22:27:39.070ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
962026-01-05T22:27:40.282ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-01-05T22:27:40.287ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
982026-01-05T22:27:40.290ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
992026-01-05T22:27:40.977ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002026-01-05T22:27:40.982ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1012026-01-05T22:27:40.984ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1022026-01-05T22:27:42.234ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-01-05T22:27:42.239ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1042026-01-05T22:27:42.242ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1052026-01-05T22:27:43.823ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-01-05T22:27:43.827ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1072026-01-05T22:27:43.830ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1082026-01-05T22:27:45.244ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-01-05T22:27:45.249ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1102026-01-05T22:27:45.252ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1112026-01-05T22:27:46.622ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-01-05T22:27:46.627ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1132026-01-05T22:27:46.630ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1142026-01-05T22:27:47.632ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
1152026-01-05T22:27:47.662ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1162026-01-05T22:27:47.663ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 20.040048059s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
1172026-01-05T22:27:47.881ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("77861647-cebf-4e85-bdde-48b1a190f0ce.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("98249b9a-2f6e-46a4-be1e-673a1bba11fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 679, tv_nsec: 948097711 } }) unit = producer-server
1182026-01-05T22:27:47.882ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1192026-01-05T22:27:47.912ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1202026-01-05T22:27:47.922ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1212026-01-05T22:27:47.932ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1222026-01-05T22:27:47.935ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1232026-01-05T22:27:48.771ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
1242026-01-05T22:27:49.324ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1252026-01-05T22:27:49.328ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1262026-01-05T22:27:49.331ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1272026-01-05T22:27:49.964ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282026-01-05T22:27:49.969ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1292026-01-05T22:27:49.971ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1302026-01-05T22:27:50.482ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312026-01-05T22:27:50.487ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1322026-01-05T22:27:50.490ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1332026-01-05T22:27:51.175ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342026-01-05T22:27:51.181ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1352026-01-05T22:27:51.183ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1362026-01-05T22:27:51.990ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-01-05T22:27:51.995ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1382026-01-05T22:27:51.998ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1392026-01-05T22:27:53.475ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-01-05T22:27:53.480ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1412026-01-05T22:27:53.482ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1422026-01-05T22:27:54.326ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-01-05T22:27:54.331ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1442026-01-05T22:27:54.333ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1452026-01-05T22:27:55.600ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-01-05T22:27:55.605ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1472026-01-05T22:27:55.608ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1482026-01-05T22:27:57.067ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-01-05T22:27:57.071ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1502026-01-05T22:27:57.074ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1512026-01-05T22:27:57.607ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
1522026-01-05T22:27:57.884ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1532026-01-05T22:27:57.889ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1542026-01-05T22:27:57.891ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1552026-01-05T22:27:58.561ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1562026-01-05T22:27:58.567ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1572026-01-05T22:27:58.569ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1582026-01-05T22:27:58.746ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
1592026-01-05T22:27:59.368ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1602026-01-05T22:27:59.373ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1612026-01-05T22:27:59.376ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1622026-01-05T22:28:00.116ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1632026-01-05T22:28:00.120ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1642026-01-05T22:28:00.123ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1652026-01-05T22:28:01.276ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1662026-01-05T22:28:01.281ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1672026-01-05T22:28:01.284ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1682026-01-05T22:28:01.954ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1692026-01-05T22:28:01.958ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1702026-01-05T22:28:01.961ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1712026-01-05T22:28:02.701ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-01-05T22:28:02.705ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1732026-01-05T22:28:02.708ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1742026-01-05T22:28:02.871ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1752026-01-05T22:28:02.873ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 35.291706362s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
1762026-01-05T22:28:03.273ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("77861647-cebf-4e85-bdde-48b1a190f0ce.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("98249b9a-2f6e-46a4-be1e-673a1bba11fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 695, tv_nsec: 382557581 } }) unit = producer-server
1772026-01-05T22:28:03.274ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
1782026-01-05T22:28:03.303ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
1792026-01-05T22:28:03.798ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1802026-01-05T22:28:03.802ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1812026-01-05T22:28:03.805ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1822026-01-05T22:28:05.028ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1832026-01-05T22:28:05.032ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1842026-01-05T22:28:05.035ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1852026-01-05T22:28:06.139ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862026-01-05T22:28:06.143ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1872026-01-05T22:28:06.145ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1882026-01-05T22:28:07.395ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-01-05T22:28:07.401ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1902026-01-05T22:28:07.404ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1912026-01-05T22:28:07.582ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
1922026-01-05T22:28:08.718ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1932026-01-05T22:28:08.721ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
1942026-01-05T22:28:08.723ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1952026-01-05T22:28:08.725ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1962026-01-05T22:28:09.483ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1972026-01-05T22:28:09.487ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
1982026-01-05T22:28:09.490ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
1992026-01-05T22:28:10.513ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-01-05T22:28:10.517ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2012026-01-05T22:28:10.520ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2022026-01-05T22:28:11.768ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032026-01-05T22:28:11.772ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2042026-01-05T22:28:11.775ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2052026-01-05T22:28:12.535ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2062026-01-05T22:28:12.540ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2072026-01-05T22:28:12.542ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2082026-01-05T22:28:13.529ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-01-05T22:28:13.533ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2102026-01-05T22:28:13.536ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2112026-01-05T22:28:14.828ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-01-05T22:28:14.833ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2132026-01-05T22:28:14.835ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2142026-01-05T22:28:16.024ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2152026-01-05T22:28:16.030ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2162026-01-05T22:28:16.033ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2172026-01-05T22:28:17.003ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2182026-01-05T22:28:17.007ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2192026-01-05T22:28:17.010ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2202026-01-05T22:28:17.558ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
2212026-01-05T22:28:18.250ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2222026-01-05T22:28:18.254ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2232026-01-05T22:28:18.257ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2242026-01-05T22:28:18.264ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2252026-01-05T22:28:18.265ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 50.725636554s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
2262026-01-05T22:28:18.696ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
2272026-01-05T22:28:18.863ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("77861647-cebf-4e85-bdde-48b1a190f0ce.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("98249b9a-2f6e-46a4-be1e-673a1bba11fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 711, tv_nsec: 15203041 } }) unit = producer-server
2282026-01-05T22:28:18.864ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
2292026-01-05T22:28:18.893ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
2302026-01-05T22:28:19.536ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2312026-01-05T22:28:19.540ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2322026-01-05T22:28:19.543ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2332026-01-05T22:28:21.012ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-01-05T22:28:21.017ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2352026-01-05T22:28:21.020ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2362026-01-05T22:28:21.744ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372026-01-05T22:28:21.748ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2382026-01-05T22:28:21.751ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2392026-01-05T22:28:22.416ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402026-01-05T22:28:22.421ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2412026-01-05T22:28:22.424ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2422026-01-05T22:28:23.711ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2432026-01-05T22:28:23.715ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2442026-01-05T22:28:23.718ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2452026-01-05T22:28:24.982ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462026-01-05T22:28:24.987ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2472026-01-05T22:28:24.990ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2482026-01-05T22:28:26.129ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2492026-01-05T22:28:26.134ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2502026-01-05T22:28:26.136ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2512026-01-05T22:28:27.492ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2522026-01-05T22:28:27.498ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2532026-01-05T22:28:27.501ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2542026-01-05T22:28:27.532ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
2552026-01-05T22:28:28.465ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2562026-01-05T22:28:28.470ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2572026-01-05T22:28:28.473ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2582026-01-05T22:28:28.671ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
2592026-01-05T22:28:29.176ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2602026-01-05T22:28:29.180ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2612026-01-05T22:28:29.183ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2622026-01-05T22:28:30.247ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2632026-01-05T22:28:30.251ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2642026-01-05T22:28:30.253ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2652026-01-05T22:28:30.985ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2662026-01-05T22:28:30.990ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2672026-01-05T22:28:30.992ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2682026-01-05T22:28:32.422ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692026-01-05T22:28:32.427ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2702026-01-05T22:28:32.430ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2712026-01-05T22:28:33.335ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722026-01-05T22:28:33.339ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2732026-01-05T22:28:33.342ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2742026-01-05T22:28:33.854ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2752026-01-05T22:28:33.855ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 66.358238812s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
2762026-01-05T22:28:34.372ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2772026-01-05T22:28:34.376ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2782026-01-05T22:28:34.379ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2792026-01-05T22:28:35.178ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2802026-01-05T22:28:35.182ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2812026-01-05T22:28:35.185ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2822026-01-05T22:28:35.949ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2832026-01-05T22:28:35.954ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2842026-01-05T22:28:35.956ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2852026-01-05T22:28:36.484ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("77861647-cebf-4e85-bdde-48b1a190f0ce.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("98249b9a-2f6e-46a4-be1e-673a1bba11fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 728, tv_nsec: 684324904 } }) unit = producer-server
2862026-01-05T22:28:36.485ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
2872026-01-05T22:28:36.515ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
2882026-01-05T22:28:37.411ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2892026-01-05T22:28:37.416ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2902026-01-05T22:28:37.419ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2912026-01-05T22:28:37.508ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
2922026-01-05T22:28:38.414ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2932026-01-05T22:28:38.419ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2942026-01-05T22:28:38.422ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2952026-01-05T22:28:38.646ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
2962026-01-05T22:28:39.622ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2972026-01-05T22:28:39.626ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
2982026-01-05T22:28:39.629ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
2992026-01-05T22:28:40.738ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3002026-01-05T22:28:40.742ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3012026-01-05T22:28:40.745ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3022026-01-05T22:28:41.272ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3032026-01-05T22:28:41.277ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3042026-01-05T22:28:41.280ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3052026-01-05T22:28:41.939ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3062026-01-05T22:28:41.943ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3072026-01-05T22:28:41.946ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3082026-01-05T22:28:42.904ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3092026-01-05T22:28:42.908ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3102026-01-05T22:28:42.911ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3112026-01-05T22:28:44.116ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3122026-01-05T22:28:44.120ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3132026-01-05T22:28:44.123ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3142026-01-05T22:28:44.683ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3152026-01-05T22:28:44.688ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3162026-01-05T22:28:44.691ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3172026-01-05T22:28:45.969ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3182026-01-05T22:28:45.974ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3192026-01-05T22:28:45.977ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3202026-01-05T22:28:46.989ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3212026-01-05T22:28:46.994ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3222026-01-05T22:28:46.997ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3232026-01-05T22:28:47.483ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
3242026-01-05T22:28:47.730ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3252026-01-05T22:28:47.734ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3262026-01-05T22:28:47.736ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3272026-01-05T22:28:48.610ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3282026-01-05T22:28:48.614ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3292026-01-05T22:28:48.617ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3302026-01-05T22:28:48.622ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
3312026-01-05T22:28:49.227ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3322026-01-05T22:28:49.232ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3332026-01-05T22:28:49.235ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3342026-01-05T22:28:50.697ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3352026-01-05T22:28:50.702ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3362026-01-05T22:28:50.705ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3372026-01-05T22:28:51.475ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3382026-01-05T22:28:51.477ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 84.027969866s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
3392026-01-05T22:28:51.684ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3402026-01-05T22:28:51.689ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3412026-01-05T22:28:51.692ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3422026-01-05T22:28:52.447ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3432026-01-05T22:28:52.451ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3442026-01-05T22:28:52.454ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3452026-01-05T22:28:53.818ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3462026-01-05T22:28:53.823ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3472026-01-05T22:28:53.826ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3482026-01-05T22:28:54.231ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("77861647-cebf-4e85-bdde-48b1a190f0ce.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("98249b9a-2f6e-46a4-be1e-673a1bba11fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 746, tv_nsec: 479789336 } }) unit = producer-server
3492026-01-05T22:28:54.232ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
3502026-01-05T22:28:54.262ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
3512026-01-05T22:28:55.290ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3522026-01-05T22:28:55.294ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3532026-01-05T22:28:55.298ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3542026-01-05T22:28:56.369ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3552026-01-05T22:28:56.374ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3562026-01-05T22:28:56.377ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3572026-01-05T22:28:57.458ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
3582026-01-05T22:28:57.691ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3592026-01-05T22:28:57.697ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3602026-01-05T22:28:57.700ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3612026-01-05T22:28:58.233ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3622026-01-05T22:28:58.237ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3632026-01-05T22:28:58.240ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3642026-01-05T22:28:58.597ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
3652026-01-05T22:28:58.894ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3662026-01-05T22:28:58.899ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3672026-01-05T22:28:58.901ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3682026-01-05T22:28:59.896ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3692026-01-05T22:28:59.900ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3702026-01-05T22:28:59.906ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3712026-01-05T22:29:01.332ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3722026-01-05T22:29:01.336ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3732026-01-05T22:29:01.339ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3742026-01-05T22:29:02.795ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3752026-01-05T22:29:02.800ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3762026-01-05T22:29:02.802ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3772026-01-05T22:29:03.541ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3782026-01-05T22:29:03.548ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3792026-01-05T22:29:03.550ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3802026-01-05T22:29:04.858ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3812026-01-05T22:29:04.862ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3822026-01-05T22:29:04.865ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3832026-01-05T22:29:06.313ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3842026-01-05T22:29:06.318ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3852026-01-05T22:29:06.320ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3862026-01-05T22:29:06.856ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3872026-01-05T22:29:06.861ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3882026-01-05T22:29:06.864ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3892026-01-05T22:29:07.434ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
3902026-01-05T22:29:08.079ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3912026-01-05T22:29:08.083ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3922026-01-05T22:29:08.089ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3932026-01-05T22:29:08.571ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
3942026-01-05T22:29:08.941ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3952026-01-05T22:29:08.945ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
3962026-01-05T22:29:08.948ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
3972026-01-05T22:29:09.222ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3982026-01-05T22:29:09.223ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 101.823025333s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
3992026-01-05T22:29:09.620ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4002026-01-05T22:29:09.625ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4012026-01-05T22:29:09.628ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4022026-01-05T22:29:10.710ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4032026-01-05T22:29:10.715ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4042026-01-05T22:29:10.718ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4052026-01-05T22:29:12.186ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4062026-01-05T22:29:12.190ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4072026-01-05T22:29:12.194ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4082026-01-05T22:29:12.816ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4092026-01-05T22:29:12.821ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4102026-01-05T22:29:12.824ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4112026-01-05T22:29:14.207ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4122026-01-05T22:29:14.211ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4132026-01-05T22:29:14.214ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4142026-01-05T22:29:14.936ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("77861647-cebf-4e85-bdde-48b1a190f0ce.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("98249b9a-2f6e-46a4-be1e-673a1bba11fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 767, tv_nsec: 241023143 } }) unit = producer-server
4152026-01-05T22:29:14.937ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
4162026-01-05T22:29:14.966ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
4172026-01-05T22:29:15.513ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4182026-01-05T22:29:15.517ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4192026-01-05T22:29:15.520ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4202026-01-05T22:29:16.332ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4212026-01-05T22:29:16.336ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4222026-01-05T22:29:16.339ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4232026-01-05T22:29:16.937ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4242026-01-05T22:29:16.941ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4252026-01-05T22:29:16.944ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4262026-01-05T22:29:17.410ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
4272026-01-05T22:29:18.276ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4282026-01-05T22:29:18.282ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4292026-01-05T22:29:18.285ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4302026-01-05T22:29:18.546ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
4312026-01-05T22:29:18.940ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4322026-01-05T22:29:18.945ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4332026-01-05T22:29:18.948ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4342026-01-05T22:29:19.801ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4352026-01-05T22:29:19.806ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4362026-01-05T22:29:19.809ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4372026-01-05T22:29:20.754ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4382026-01-05T22:29:20.759ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4392026-01-05T22:29:20.762ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4402026-01-05T22:29:21.547ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4412026-01-05T22:29:21.551ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4422026-01-05T22:29:21.554ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4432026-01-05T22:29:22.067ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4442026-01-05T22:29:22.071ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4452026-01-05T22:29:22.074ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4462026-01-05T22:29:23.313ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4472026-01-05T22:29:23.317ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4482026-01-05T22:29:23.321ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4492026-01-05T22:29:24.294ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4502026-01-05T22:29:24.298ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4512026-01-05T22:29:24.301ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4522026-01-05T22:29:25.192ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4532026-01-05T22:29:25.197ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4542026-01-05T22:29:25.200ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4552026-01-05T22:29:26.692ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4562026-01-05T22:29:26.697ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4572026-01-05T22:29:26.699ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4582026-01-05T22:29:27.385ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
4592026-01-05T22:29:27.948ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4602026-01-05T22:29:27.952ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4612026-01-05T22:29:27.955ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4622026-01-05T22:29:28.521ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
4632026-01-05T22:29:29.025ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4642026-01-05T22:29:29.029ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4652026-01-05T22:29:29.032ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4662026-01-05T22:29:29.926ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4672026-01-05T22:29:29.927ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 122.583402901s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
4682026-01-05T22:29:29.968ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4692026-01-05T22:29:29.973ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4702026-01-05T22:29:29.976ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4712026-01-05T22:29:30.541ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4722026-01-05T22:29:30.546ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4732026-01-05T22:29:30.549ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4742026-01-05T22:29:31.243ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4752026-01-05T22:29:31.247ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4762026-01-05T22:29:31.250ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4772026-01-05T22:29:32.001ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4782026-01-05T22:29:32.005ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4792026-01-05T22:29:32.007ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4802026-01-05T22:29:32.832ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4812026-01-05T22:29:32.838ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4822026-01-05T22:29:32.840ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4832026-01-05T22:29:33.491ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4842026-01-05T22:29:33.495ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4852026-01-05T22:29:33.498ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4862026-01-05T22:29:34.809ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4872026-01-05T22:29:34.814ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4882026-01-05T22:29:34.816ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4892026-01-05T22:29:35.806ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4902026-01-05T22:29:35.811ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4912026-01-05T22:29:35.814ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4922026-01-05T22:29:36.330ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4932026-01-05T22:29:36.334ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4942026-01-05T22:29:36.337ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4952026-01-05T22:29:37.121ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4962026-01-05T22:29:37.127ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
4972026-01-05T22:29:37.129ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
4982026-01-05T22:29:37.360ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
4992026-01-05T22:29:38.496ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
5002026-01-05T22:29:38.580ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5012026-01-05T22:29:38.585ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5022026-01-05T22:29:38.587ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5032026-01-05T22:29:39.683ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5042026-01-05T22:29:39.687ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5052026-01-05T22:29:39.690ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5062026-01-05T22:29:40.618ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5072026-01-05T22:29:40.623ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5082026-01-05T22:29:40.626ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5092026-01-05T22:29:41.357ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5102026-01-05T22:29:41.361ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5112026-01-05T22:29:41.365ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5122026-01-05T22:29:41.889ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5132026-01-05T22:29:41.894ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5142026-01-05T22:29:41.896ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5152026-01-05T22:29:42.936ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5162026-01-05T22:29:42.940ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5172026-01-05T22:29:42.943ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5182026-01-05T22:29:43.995ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5192026-01-05T22:29:44.000ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5202026-01-05T22:29:44.003ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5212026-01-05T22:29:44.976ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5222026-01-05T22:29:44.980ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5232026-01-05T22:29:44.983ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5242026-01-05T22:29:45.769ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5252026-01-05T22:29:45.773ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5262026-01-05T22:29:45.776ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5272026-01-05T22:29:46.737ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5282026-01-05T22:29:46.741ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5292026-01-05T22:29:46.744ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5302026-01-05T22:29:47.336ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
5312026-01-05T22:29:47.840ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5322026-01-05T22:29:47.844ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5332026-01-05T22:29:47.847ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5342026-01-05T22:29:48.470ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
5352026-01-05T22:29:48.476ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5362026-01-05T22:29:48.480ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5372026-01-05T22:29:48.483ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5382026-01-05T22:29:49.176ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("77861647-cebf-4e85-bdde-48b1a190f0ce.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("98249b9a-2f6e-46a4-be1e-673a1bba11fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a85b9783-de7f-4744-8cf3-ac8a31b0cd75.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 801, tv_nsec: 573689741 } }) unit = producer-server
5392026-01-05T22:29:49.176ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221 unit = producer-server
5402026-01-05T22:29:49.206ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
5412026-01-05T22:29:49.856ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5422026-01-05T22:29:49.860ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5432026-01-05T22:29:49.863ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5442026-01-05T22:29:51.355ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5452026-01-05T22:29:51.359ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5462026-01-05T22:29:51.362ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5472026-01-05T22:29:51.904ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5482026-01-05T22:29:51.908ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5492026-01-05T22:29:51.911ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5502026-01-05T22:29:52.476ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5512026-01-05T22:29:52.480ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5522026-01-05T22:29:52.483ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5532026-01-05T22:29:53.367ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5542026-01-05T22:29:53.372ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5552026-01-05T22:29:53.375ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5562026-01-05T22:29:54.411ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5572026-01-05T22:29:54.415ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5582026-01-05T22:29:54.418ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5592026-01-05T22:29:55.108ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5602026-01-05T22:29:55.112ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5612026-01-05T22:29:55.115ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5622026-01-05T22:29:55.654ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5632026-01-05T22:29:55.659ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5642026-01-05T22:29:55.661ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5652026-01-05T22:29:56.508ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5662026-01-05T22:29:56.513ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5672026-01-05T22:29:56.515ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5682026-01-05T22:29:57.311ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 7736234662763567125 n_samples = 6 unit = oximeter
5692026-01-05T22:29:57.694ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5702026-01-05T22:29:57.698ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5712026-01-05T22:29:57.701ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5722026-01-05T22:29:58.446ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5090599079127267502 n_samples = 6 unit = oximeter
5732026-01-05T22:29:58.801ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5742026-01-05T22:29:58.805ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5752026-01-05T22:29:58.808ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]
5762026-01-05T22:30:00.001ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5772026-01-05T22:30:00.006ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::24f8:78ff:fe78:55d4]
5782026-01-05T22:30:00.009ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.1.174]