Max level shown:
1 [ Feb 3 21:12:34 Enabled. ]
2 [ Feb 3 21:12:34 Rereading configuration. ]
3 [ Feb 3 21:12:35 Rereading configuration. ]
4 [ Feb 3 21:12:56 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Feb 3 21:12:56 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-02-03T21:12:56.490ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-02-03T21:12:56.536ZINFOtfportd: connected to dpd running softnpu-9f6b8f71
92026-02-03T21:12:56.536ZINFOtfportd: running in sync-only mode
102026-02-03T21:12:56.538ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-02-03T21:12:56.539ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-02-03T21:12:56.540ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 31b37f59-785f-4b61-9799-d7ae1d038657, ), sled_id: Some( b22e8cfe-0426-411f-8dc8-167b022c87d8, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
142026-02-03T21:12:56.540ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-02-03T21:12:56.544ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::703e:4ff:feea:9987->72:3e:04:ea:99:87 interface = tfportrear0_0
162026-02-03T21:12:56.549ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-02-03T21:12:56.554ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
182026-02-03T21:12:56.559ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Feb 3 21:12:57 Rereading configuration. ]
20 [ Feb 3 21:12:57 Executing refresh method (:kill). ]
21 [ Feb 3 21:12:57 Stopping because all processes in service exited. ]
22 [ Feb 3 21:12:57 Executing stop method (:kill). ]
23 [ Feb 3 21:12:57 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Feb 3 21:12:57 Method "start" exited with status 0. ]
252026-02-03T21:12:57.064ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-02-03T21:12:57.106ZINFOtfportd: connected to dpd running softnpu-9f6b8f71
272026-02-03T21:12:57.106ZINFOtfportd: running in sync-only mode
282026-02-03T21:12:57.107ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-02-03T21:12:57.108ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-02-03T21:12:57.108ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 31b37f59-785f-4b61-9799-d7ae1d038657, sled_id: b22e8cfe-0426-411f-8dc8-167b022c87d8, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, lot: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 68a56cff-6937-4050-b8c4-787285681195, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-02-03T21:12:57.108ZDEBGtfportd: created producer ID producer_id = eb389f58-154e-4a2e-a295-b9a79a47101a unit = oximeter
322026-02-03T21:12:57.110ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 31b37f59-785f-4b61-9799-d7ae1d038657, ), sled_id: Some( b22e8cfe-0426-411f-8dc8-167b022c87d8, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
332026-02-03T21:12:57.116ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-02-03T21:12:57.118ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::703e:4ff:feea:9987->72:3e:04:ea:99:87 interface = tfportrear0_0
352026-02-03T21:12:57.120ZDEBGtfportd: registered DTrace probes unit = producer-server
362026-02-03T21:12:57.121ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
372026-02-03T21:12:57.122ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:35373 method = GET path = /{producer_id} unit = producer-server versions = all
382026-02-03T21:12:57.122ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:35373 unit = producer-server
392026-02-03T21:12:57.122ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:35373 unit = producer-server
402026-02-03T21:12:57.122ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:35373 unit = producer-server
412026-02-03T21:12:57.122ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-02-03T21:12:57.123ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-02-03T21:12:57.124ZDEBGtfportd: starting producer registration task unit = producer-server
442026-02-03T21:12:57.124ZWARNtfportd: tfportqsfp0_0 has no addrs
452026-02-03T21:12:57.124ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:35373 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = eb389f58-154e-4a2e-a295-b9a79a47101a unit = producer-server
462026-02-03T21:12:57.124ZINFOtfportd: created oximeter producer server unit = oximeter
472026-02-03T21:12:57.124ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
482026-02-03T21:12:57.124ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-02-03T21:12:57.124ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-02-03T21:12:30.258798098Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502026-02-03T21:12:57.124ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 14737799007253439550 unit = oximeter
512026-02-03T21:12:57.124ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522026-02-03T21:12:57.125ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 1.32417ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
532026-02-03T21:12:57.379ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 254.869877ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
542026-02-03T21:12:57.794ZINFOtfportd: added new ARP entry entry = 10.151.2.174->8e:13:da:8c:da:ac interface = tfportqsfp0_0
552026-02-03T21:12:57.892ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 767.719186ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
562026-02-03T21:12:58.385ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
572026-02-03T21:12:58.390ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
582026-02-03T21:12:58.393ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
592026-02-03T21:12:58.393ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
602026-02-03T21:12:58.395ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-02-03T21:12:30.258662335Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
612026-02-03T21:12:58.395ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 5830328683346957500 unit = oximeter
622026-02-03T21:12:58.395ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
632026-02-03T21:12:58.491ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::60ab:49ff:fe52:e8a1->62:ab:49:52:e8:a1 interface = tfportrear0_0
642026-02-03T21:12:59.373ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("700aed7f-ae1c-4ac5-9739-c91e186f8b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("753fc4fb-3ac2-4233-9333-1ec564b3e388.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9082d449-b202-4be7-a66a-77237ac91b25.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9082d449-b202-4be7-a66a-77237ac91b25.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 685, tv_nsec: 320620679 } }) unit = producer-server
652026-02-03T21:12:59.374ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
662026-02-03T21:12:59.408ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
672026-02-03T21:12:59.611ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
682026-02-03T21:12:59.615ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
692026-02-03T21:12:59.618ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
702026-02-03T21:13:00.266ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-02-03T21:13:00.271ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
722026-02-03T21:13:00.274ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
732026-02-03T21:13:01.756ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-02-03T21:13:01.760ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
752026-02-03T21:13:01.764ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
762026-02-03T21:13:02.593ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-02-03T21:13:02.598ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
782026-02-03T21:13:02.600ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
792026-02-03T21:13:03.846ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-02-03T21:13:03.850ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
812026-02-03T21:13:03.853ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
822026-02-03T21:13:05.080ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-02-03T21:13:05.085ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
842026-02-03T21:13:05.088ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
852026-02-03T21:13:05.943ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862026-02-03T21:13:05.948ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
872026-02-03T21:13:05.950ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
882026-02-03T21:13:07.006ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
892026-02-03T21:13:07.011ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
902026-02-03T21:13:07.014ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
912026-02-03T21:13:07.128ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
922026-02-03T21:13:08.317ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
932026-02-03T21:13:08.322ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
942026-02-03T21:13:08.325ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
952026-02-03T21:13:08.397ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
962026-02-03T21:13:09.042ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-02-03T21:13:09.047ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
982026-02-03T21:13:09.050ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
992026-02-03T21:13:09.642ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002026-02-03T21:13:09.647ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1012026-02-03T21:13:09.650ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1022026-02-03T21:13:10.326ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-02-03T21:13:10.330ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1042026-02-03T21:13:10.333ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1052026-02-03T21:13:11.538ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-02-03T21:13:11.542ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1072026-02-03T21:13:11.545ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1082026-02-03T21:13:12.411ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-02-03T21:13:12.415ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1102026-02-03T21:13:12.418ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1112026-02-03T21:13:13.152ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-02-03T21:13:13.156ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1132026-02-03T21:13:13.160ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1142026-02-03T21:13:14.288ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1152026-02-03T21:13:14.297ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1162026-02-03T21:13:14.300ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1172026-02-03T21:13:14.418ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1182026-02-03T21:13:14.419ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 17.285918345s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1192026-02-03T21:13:15.607ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1202026-02-03T21:13:15.613ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1212026-02-03T21:13:15.615ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1222026-02-03T21:13:16.412ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("700aed7f-ae1c-4ac5-9739-c91e186f8b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("753fc4fb-3ac2-4233-9333-1ec564b3e388.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9082d449-b202-4be7-a66a-77237ac91b25.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9082d449-b202-4be7-a66a-77237ac91b25.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 702, tv_nsec: 346201642 } }) unit = producer-server
1232026-02-03T21:13:16.412ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1242026-02-03T21:13:16.448ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1252026-02-03T21:13:16.598ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1262026-02-03T21:13:16.603ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1272026-02-03T21:13:16.606ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1282026-02-03T21:13:17.146ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
1292026-02-03T21:13:17.966ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1302026-02-03T21:13:17.987ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1312026-02-03T21:13:18.001ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1322026-02-03T21:13:18.421ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
1332026-02-03T21:13:18.654ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342026-02-03T21:13:18.676ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1352026-02-03T21:13:18.692ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1362026-02-03T21:13:19.920ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-02-03T21:13:19.926ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1382026-02-03T21:13:19.930ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1392026-02-03T21:13:21.091ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-02-03T21:13:21.095ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1412026-02-03T21:13:21.098ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1422026-02-03T21:13:22.073ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-02-03T21:13:22.078ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1442026-02-03T21:13:22.081ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1452026-02-03T21:13:23.051ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-02-03T21:13:23.056ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1472026-02-03T21:13:23.058ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1482026-02-03T21:13:23.679ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1492026-02-03T21:13:23.684ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1502026-02-03T21:13:23.687ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1512026-02-03T21:13:24.664ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-02-03T21:13:24.669ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1532026-02-03T21:13:24.673ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1542026-02-03T21:13:26.031ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1552026-02-03T21:13:26.035ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1562026-02-03T21:13:26.037ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1572026-02-03T21:13:27.177ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
1582026-02-03T21:13:27.526ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1592026-02-03T21:13:27.531ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1602026-02-03T21:13:27.534ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1612026-02-03T21:13:28.359ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1622026-02-03T21:13:28.365ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1632026-02-03T21:13:28.368ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1642026-02-03T21:13:28.448ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
1652026-02-03T21:13:29.535ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1662026-02-03T21:13:29.540ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1672026-02-03T21:13:29.543ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1682026-02-03T21:13:30.146ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1692026-02-03T21:13:30.152ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1702026-02-03T21:13:30.155ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1712026-02-03T21:13:30.898ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-02-03T21:13:30.905ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1732026-02-03T21:13:30.909ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1742026-02-03T21:13:31.596ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1752026-02-03T21:13:31.596ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 34.322809709s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1762026-02-03T21:13:31.906ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1772026-02-03T21:13:31.912ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1782026-02-03T21:13:31.915ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1792026-02-03T21:13:33.042ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1802026-02-03T21:13:33.051ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1812026-02-03T21:13:33.055ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1822026-02-03T21:13:33.789ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("700aed7f-ae1c-4ac5-9739-c91e186f8b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("753fc4fb-3ac2-4233-9333-1ec564b3e388.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9082d449-b202-4be7-a66a-77237ac91b25.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9082d449-b202-4be7-a66a-77237ac91b25.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 719, tv_nsec: 676109653 } }) unit = producer-server
1832026-02-03T21:13:33.789ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
1842026-02-03T21:13:33.824ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
1852026-02-03T21:13:33.850ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1862026-02-03T21:13:33.855ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1872026-02-03T21:13:33.858ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1882026-02-03T21:13:34.467ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-02-03T21:13:34.472ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1902026-02-03T21:13:34.476ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1912026-02-03T21:13:35.192ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1922026-02-03T21:13:35.197ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1932026-02-03T21:13:35.200ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1942026-02-03T21:13:36.542ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1952026-02-03T21:13:36.547ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
1962026-02-03T21:13:36.550ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1972026-02-03T21:13:37.206ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
1982026-02-03T21:13:37.913ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1992026-02-03T21:13:37.918ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2002026-02-03T21:13:37.921ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2012026-02-03T21:13:38.478ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
2022026-02-03T21:13:38.876ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2032026-02-03T21:13:38.880ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2042026-02-03T21:13:38.883ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2052026-02-03T21:13:39.873ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2062026-02-03T21:13:39.878ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2072026-02-03T21:13:39.881ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2082026-02-03T21:13:41.390ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2092026-02-03T21:13:41.395ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2102026-02-03T21:13:41.398ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2112026-02-03T21:13:42.080ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-02-03T21:13:42.086ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2132026-02-03T21:13:42.090ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2142026-02-03T21:13:43.585ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2152026-02-03T21:13:43.591ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2162026-02-03T21:13:43.595ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2172026-02-03T21:13:44.443ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2182026-02-03T21:13:44.449ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2192026-02-03T21:13:44.452ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2202026-02-03T21:13:45.374ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2212026-02-03T21:13:45.379ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2222026-02-03T21:13:45.382ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2232026-02-03T21:13:45.966ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2242026-02-03T21:13:45.970ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2252026-02-03T21:13:45.974ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2262026-02-03T21:13:47.205ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2272026-02-03T21:13:47.212ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2282026-02-03T21:13:47.215ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2292026-02-03T21:13:47.237ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
2302026-02-03T21:13:47.843ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2312026-02-03T21:13:47.848ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2322026-02-03T21:13:47.851ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2332026-02-03T21:13:48.429ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-02-03T21:13:48.434ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2352026-02-03T21:13:48.437ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2362026-02-03T21:13:48.506ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
2372026-02-03T21:13:48.866ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2382026-02-03T21:13:48.866ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 51.642015376s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2392026-02-03T21:13:49.716ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2402026-02-03T21:13:49.721ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2412026-02-03T21:13:49.725ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2422026-02-03T21:13:51.147ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2432026-02-03T21:13:51.153ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2442026-02-03T21:13:51.156ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2452026-02-03T21:13:52.098ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2462026-02-03T21:13:52.102ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2472026-02-03T21:13:52.105ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2482026-02-03T21:13:53.187ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2492026-02-03T21:13:53.191ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2502026-02-03T21:13:53.194ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2512026-02-03T21:13:54.562ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2522026-02-03T21:13:54.567ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2532026-02-03T21:13:54.570ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2542026-02-03T21:13:55.324ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2552026-02-03T21:13:55.329ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2562026-02-03T21:13:55.331ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2572026-02-03T21:13:56.408ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2582026-02-03T21:13:56.414ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2592026-02-03T21:13:56.417ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2602026-02-03T21:13:57.265ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
2612026-02-03T21:13:57.448ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("700aed7f-ae1c-4ac5-9739-c91e186f8b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("753fc4fb-3ac2-4233-9333-1ec564b3e388.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9082d449-b202-4be7-a66a-77237ac91b25.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9082d449-b202-4be7-a66a-77237ac91b25.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 743, tv_nsec: 271963525 } }) unit = producer-server
2622026-02-03T21:13:57.448ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
2632026-02-03T21:13:57.480ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
2642026-02-03T21:13:57.737ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2652026-02-03T21:13:57.742ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2662026-02-03T21:13:57.746ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2672026-02-03T21:13:58.535ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
2682026-02-03T21:13:58.937ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692026-02-03T21:13:58.942ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2702026-02-03T21:13:58.945ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2712026-02-03T21:13:59.608ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722026-02-03T21:13:59.613ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2732026-02-03T21:13:59.616ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2742026-02-03T21:14:00.937ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2752026-02-03T21:14:00.943ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2762026-02-03T21:14:00.946ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2772026-02-03T21:14:01.907ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2782026-02-03T21:14:01.913ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2792026-02-03T21:14:01.916ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2802026-02-03T21:14:03.319ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2812026-02-03T21:14:03.324ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2822026-02-03T21:14:03.327ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2832026-02-03T21:14:04.402ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2842026-02-03T21:14:04.407ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2852026-02-03T21:14:04.411ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2862026-02-03T21:14:05.549ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2872026-02-03T21:14:05.554ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2882026-02-03T21:14:05.558ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2892026-02-03T21:14:06.936ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2902026-02-03T21:14:06.941ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2912026-02-03T21:14:06.944ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2922026-02-03T21:14:07.293ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
2932026-02-03T21:14:08.451ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2942026-02-03T21:14:08.456ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2952026-02-03T21:14:08.460ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2962026-02-03T21:14:08.563ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
2972026-02-03T21:14:09.086ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2982026-02-03T21:14:09.093ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
2992026-02-03T21:14:09.096ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3002026-02-03T21:14:10.279ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3012026-02-03T21:14:10.284ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3022026-02-03T21:14:10.287ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3032026-02-03T21:14:11.347ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3042026-02-03T21:14:11.352ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3052026-02-03T21:14:11.355ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3062026-02-03T21:14:12.341ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3072026-02-03T21:14:12.346ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3082026-02-03T21:14:12.349ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3092026-02-03T21:14:12.522ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3102026-02-03T21:14:12.523ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 75.234993352s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3112026-02-03T21:14:12.926ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3122026-02-03T21:14:12.931ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3132026-02-03T21:14:12.935ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3142026-02-03T21:14:14.127ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3152026-02-03T21:14:14.132ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3162026-02-03T21:14:14.135ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3172026-02-03T21:14:14.738ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3182026-02-03T21:14:14.742ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3192026-02-03T21:14:14.745ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3202026-02-03T21:14:15.863ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3212026-02-03T21:14:15.868ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3222026-02-03T21:14:15.871ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3232026-02-03T21:14:16.966ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3242026-02-03T21:14:16.976ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3252026-02-03T21:14:16.979ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3262026-02-03T21:14:17.323ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
3272026-02-03T21:14:17.884ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3282026-02-03T21:14:17.890ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3292026-02-03T21:14:17.892ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3302026-02-03T21:14:18.592ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
3312026-02-03T21:14:19.355ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3322026-02-03T21:14:19.361ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3332026-02-03T21:14:19.364ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3342026-02-03T21:14:20.739ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3352026-02-03T21:14:20.745ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3362026-02-03T21:14:20.748ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3372026-02-03T21:14:21.326ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382026-02-03T21:14:21.331ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3392026-02-03T21:14:21.334ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3402026-02-03T21:14:22.355ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3412026-02-03T21:14:22.361ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3422026-02-03T21:14:22.364ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3432026-02-03T21:14:22.570ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("700aed7f-ae1c-4ac5-9739-c91e186f8b0f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("753fc4fb-3ac2-4233-9333-1ec564b3e388.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("9082d449-b202-4be7-a66a-77237ac91b25.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9082d449-b202-4be7-a66a-77237ac91b25.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::a)) }], valid_until: Instant { tv_sec: 768, tv_nsec: 326926812 } }) unit = producer-server
3442026-02-03T21:14:22.571ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221 unit = producer-server
3452026-02-03T21:14:22.607ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
3462026-02-03T21:14:23.459ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3472026-02-03T21:14:23.470ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3482026-02-03T21:14:23.475ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3492026-02-03T21:14:32.586ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3502026-02-03T21:14:32.626ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3512026-02-03T21:14:32.741ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3522026-02-03T21:14:32.768ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
3532026-02-03T21:14:32.771ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
3542026-02-03T21:14:34.158ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3552026-02-03T21:14:34.171ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3562026-02-03T21:14:34.192ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3572026-02-03T21:14:35.107ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3582026-02-03T21:14:35.116ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3592026-02-03T21:14:35.128ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3602026-02-03T21:14:35.807ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3612026-02-03T21:14:35.820ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3622026-02-03T21:14:35.831ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3632026-02-03T21:14:36.940ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3642026-02-03T21:14:36.949ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3652026-02-03T21:14:36.954ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3662026-02-03T21:14:37.661ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3672026-02-03T21:14:37.698ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 7 delay = 100.341548012s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3682026-02-03T21:14:38.028ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3692026-02-03T21:14:38.036ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3702026-02-03T21:14:38.045ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3712026-02-03T21:14:39.528ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3722026-02-03T21:14:39.535ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3732026-02-03T21:14:39.540ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3742026-02-03T21:14:40.272ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3752026-02-03T21:14:40.279ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3762026-02-03T21:14:40.288ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3772026-02-03T21:14:41.268ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3782026-02-03T21:14:41.275ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3792026-02-03T21:14:41.279ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3802026-02-03T21:14:42.741ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3812026-02-03T21:14:42.747ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3822026-02-03T21:14:42.751ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3832026-02-03T21:14:42.799ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
3842026-02-03T21:14:42.799ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
3852026-02-03T21:14:43.787ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3862026-02-03T21:14:43.794ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3872026-02-03T21:14:43.799ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3882026-02-03T21:14:44.360ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3892026-02-03T21:14:44.367ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3902026-02-03T21:14:44.370ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3912026-02-03T21:14:45.010ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3922026-02-03T21:14:45.015ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3932026-02-03T21:14:45.018ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3942026-02-03T21:14:45.912ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3952026-02-03T21:14:45.918ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3962026-02-03T21:14:45.921ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3972026-02-03T21:14:46.959ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3982026-02-03T21:14:46.965ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
3992026-02-03T21:14:46.968ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4002026-02-03T21:14:48.299ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4012026-02-03T21:14:48.304ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4022026-02-03T21:14:48.307ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4032026-02-03T21:14:49.529ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4042026-02-03T21:14:49.537ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4052026-02-03T21:14:49.540ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4062026-02-03T21:14:51.001ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4072026-02-03T21:14:51.009ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4082026-02-03T21:14:51.013ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4092026-02-03T21:14:51.869ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4102026-02-03T21:14:51.875ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4112026-02-03T21:14:51.879ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4122026-02-03T21:14:52.799ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 14737799007253439550 n_samples = 6 unit = oximeter
4132026-02-03T21:14:52.800ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 5830328683346957500 n_samples = 6 unit = oximeter
4142026-02-03T21:14:53.223ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4152026-02-03T21:14:53.228ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4162026-02-03T21:14:53.231ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4172026-02-03T21:14:53.839ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4182026-02-03T21:14:53.845ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4192026-02-03T21:14:53.848ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4202026-02-03T21:14:54.801ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4212026-02-03T21:14:54.808ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4222026-02-03T21:14:54.812ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4232026-02-03T21:14:55.346ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4242026-02-03T21:14:55.354ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4252026-02-03T21:14:55.359ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4262026-02-03T21:14:56.059ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4272026-02-03T21:14:56.067ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4282026-02-03T21:14:56.070ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4292026-02-03T21:14:57.146ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4302026-02-03T21:14:57.151ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4312026-02-03T21:14:57.154ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4322026-02-03T21:14:58.545ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4332026-02-03T21:14:58.551ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4342026-02-03T21:14:58.553ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4352026-02-03T21:14:59.891ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4362026-02-03T21:14:59.897ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4372026-02-03T21:14:59.901ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4382026-02-03T21:15:01.400ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4392026-02-03T21:15:01.409ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::703e:4ff:feea:9987]
4402026-02-03T21:15:01.414ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]