Max level shown:
1 [ Jan 30 21:12:10 Enabled. ]
2 [ Jan 30 21:12:10 Rereading configuration. ]
3 [ Jan 30 21:12:11 Rereading configuration. ]
4 [ Jan 30 21:12:22 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jan 30 21:12:22 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-01-30T21:12:22.636ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-01-30T21:12:22.679ZINFOtfportd: connected to dpd running softnpu-846348bf
92026-01-30T21:12:22.679ZINFOtfportd: running in sync-only mode
102026-01-30T21:12:22.679ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-01-30T21:12:22.681ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-01-30T21:12:22.683ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( cb7c3a03-6bbc-4c19-bcb7-7717e25de20b, ), sled_id: Some( f0b94cfd-cc4b-46a7-af85-efb3852617f1, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
142026-01-30T21:12:22.683ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-01-30T21:12:22.688ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::bcbb:3fff:fe6e:41e7->be:bb:3f:6e:41:e7 interface = tfportrear0_0
162026-01-30T21:12:22.688ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-01-30T21:12:22.693ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
182026-01-30T21:12:22.698ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Jan 30 21:12:23 Rereading configuration. ]
20 [ Jan 30 21:12:23 Executing refresh method (:kill). ]
21 [ Jan 30 21:12:23 Stopping because all processes in service exited. ]
22 [ Jan 30 21:12:23 Executing stop method (:kill). ]
23 [ Jan 30 21:12:23 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Jan 30 21:12:23 Method "start" exited with status 0. ]
252026-01-30T21:12:23.337ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-01-30T21:12:23.387ZINFOtfportd: connected to dpd running softnpu-846348bf
272026-01-30T21:12:23.387ZINFOtfportd: running in sync-only mode
282026-01-30T21:12:23.387ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-01-30T21:12:23.389ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-01-30T21:12:23.389ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: cb7c3a03-6bbc-4c19-bcb7-7717e25de20b, sled_id: f0b94cfd-cc4b-46a7-af85-efb3852617f1, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, lot: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 60c6d0c1-f44b-4404-b432-c4a3e33e9b3d, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-01-30T21:12:23.389ZDEBGtfportd: created producer ID producer_id = 5b03a63f-53bc-4eba-a700-5c0784456ec7 unit = oximeter
322026-01-30T21:12:23.392ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( cb7c3a03-6bbc-4c19-bcb7-7717e25de20b, ), sled_id: Some( f0b94cfd-cc4b-46a7-af85-efb3852617f1, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
332026-01-30T21:12:23.397ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-01-30T21:12:23.399ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::bcbb:3fff:fe6e:41e7->be:bb:3f:6e:41:e7 interface = tfportrear0_0
352026-01-30T21:12:23.399ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::7427:2aff:fe93:55e4->76:27:2a:93:55:e4 interface = tfportrear0_0
362026-01-30T21:12:23.401ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-01-30T21:12:23.402ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:37579 method = GET path = /{producer_id} unit = producer-server versions = all
382026-01-30T21:12:23.402ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:37579 unit = producer-server
392026-01-30T21:12:23.402ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:37579 unit = producer-server
402026-01-30T21:12:23.402ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:37579 unit = producer-server
412026-01-30T21:12:23.402ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-01-30T21:12:23.402ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-01-30T21:12:23.402ZDEBGtfportd: starting producer registration task unit = producer-server
442026-01-30T21:12:23.402ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:37579 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 5b03a63f-53bc-4eba-a700-5c0784456ec7 unit = producer-server
452026-01-30T21:12:23.402ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
462026-01-30T21:12:23.402ZINFOtfportd: created oximeter producer server unit = oximeter
472026-01-30T21:12:23.403ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
482026-01-30T21:12:23.403ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-01-30T21:12:23.403ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-01-30T21:12:06.643416857Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
502026-01-30T21:12:23.404ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 11886258443777427216 unit = oximeter
512026-01-30T21:12:23.404ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
522026-01-30T21:12:23.406ZWARNtfportd: tfportqsfp0_0 has no addrs
532026-01-30T21:12:24.437ZINFOtfportd: added new ARP entry entry = 10.151.2.174->66:f4:66:42:8c:14 interface = tfportqsfp0_0
542026-01-30T21:12:24.694ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
552026-01-30T21:12:24.700ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
562026-01-30T21:12:24.702ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
572026-01-30T21:12:24.703ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
582026-01-30T21:12:24.704ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-01-30T21:12:06.643264414Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
592026-01-30T21:12:24.704ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 972073342625437383 unit = oximeter
602026-01-30T21:12:24.704ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
612026-01-30T21:12:25.959ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
622026-01-30T21:12:25.966ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
632026-01-30T21:12:25.969ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
642026-01-30T21:12:27.476ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
652026-01-30T21:12:27.481ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
662026-01-30T21:12:27.484ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
672026-01-30T21:12:28.405ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3cd0169a-e134-4091-812e-538f3f58ea17.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 655, tv_nsec: 980001452 } }) unit = producer-server
682026-01-30T21:12:28.405ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
692026-01-30T21:12:28.438ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
702026-01-30T21:12:28.438ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
712026-01-30T21:12:28.444ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
722026-01-30T21:12:28.447ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
732026-01-30T21:12:28.972ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
742026-01-30T21:12:28.981ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
752026-01-30T21:12:28.983ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
762026-01-30T21:12:29.617ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
772026-01-30T21:12:29.621ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
782026-01-30T21:12:29.624ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
792026-01-30T21:12:30.568ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
802026-01-30T21:12:30.573ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
812026-01-30T21:12:30.576ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
822026-01-30T21:12:31.887ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
832026-01-30T21:12:31.892ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
842026-01-30T21:12:31.895ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
852026-01-30T21:12:33.040ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
862026-01-30T21:12:33.045ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
872026-01-30T21:12:33.048ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
882026-01-30T21:12:33.408ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
892026-01-30T21:12:34.290ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902026-01-30T21:12:34.295ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
912026-01-30T21:12:34.298ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
922026-01-30T21:12:34.705ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
932026-01-30T21:12:35.533ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942026-01-30T21:12:35.538ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
952026-01-30T21:12:35.540ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
962026-01-30T21:12:36.833ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
972026-01-30T21:12:36.838ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
982026-01-30T21:12:36.840ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
992026-01-30T21:12:37.718ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1002026-01-30T21:12:37.722ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1012026-01-30T21:12:37.725ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1022026-01-30T21:12:39.200ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1032026-01-30T21:12:39.205ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1042026-01-30T21:12:39.208ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1052026-01-30T21:12:40.082ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1062026-01-30T21:12:40.087ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1072026-01-30T21:12:40.091ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1082026-01-30T21:12:41.156ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1092026-01-30T21:12:41.161ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1102026-01-30T21:12:41.164ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1112026-01-30T21:12:42.289ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-01-30T21:12:42.294ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1132026-01-30T21:12:42.297ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1142026-01-30T21:12:42.974ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1152026-01-30T21:12:42.976ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 20.050934394s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
1162026-01-30T21:12:42.976ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
1172026-01-30T21:12:42.991ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1182026-01-30T21:12:42.997ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1192026-01-30T21:12:43.003ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1202026-01-30T21:12:43.110ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3cd0169a-e134-4091-812e-538f3f58ea17.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 671, tv_nsec: 162758879 } }) unit = producer-server
1212026-01-30T21:12:43.132ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
1222026-01-30T21:12:43.654ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1232026-01-30T21:12:43.907ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1242026-01-30T21:12:43.915ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1252026-01-30T21:12:43.920ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1262026-01-30T21:12:44.233ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
1272026-01-30T21:12:45.160ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282026-01-30T21:12:45.166ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1292026-01-30T21:12:45.170ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1302026-01-30T21:12:46.631ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1312026-01-30T21:12:46.637ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1322026-01-30T21:12:46.640ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1332026-01-30T21:12:48.118ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1342026-01-30T21:12:48.123ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1352026-01-30T21:12:48.126ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1362026-01-30T21:12:49.621ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1372026-01-30T21:12:49.626ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1382026-01-30T21:12:49.628ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1392026-01-30T21:12:50.366ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1402026-01-30T21:12:50.371ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1412026-01-30T21:12:50.373ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1422026-01-30T21:12:51.650ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1432026-01-30T21:12:51.655ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1442026-01-30T21:12:51.659ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1452026-01-30T21:12:52.867ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1462026-01-30T21:12:52.872ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1472026-01-30T21:12:52.874ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1482026-01-30T21:12:53.007ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
1492026-01-30T21:12:54.174ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1502026-01-30T21:12:54.179ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1512026-01-30T21:12:54.182ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1522026-01-30T21:12:54.262ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
1532026-01-30T21:12:55.542ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1542026-01-30T21:12:55.547ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1552026-01-30T21:12:55.550ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1562026-01-30T21:12:56.882ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1572026-01-30T21:12:56.886ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1582026-01-30T21:12:56.889ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1592026-01-30T21:12:57.785ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1602026-01-30T21:12:57.790ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1612026-01-30T21:12:57.793ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1622026-01-30T21:12:58.681ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1632026-01-30T21:12:58.687ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1642026-01-30T21:12:58.691ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1652026-01-30T21:12:58.697ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1662026-01-30T21:12:58.699ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 35.731283995s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
1672026-01-30T21:13:01.711ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1682026-01-30T21:13:01.981ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1692026-01-30T21:13:01.984ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3cd0169a-e134-4091-812e-538f3f58ea17.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 689, tv_nsec: 992533075 } }) unit = producer-server
1702026-01-30T21:13:01.989ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1712026-01-30T21:13:01.992ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
1722026-01-30T21:13:02.310ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1732026-01-30T21:13:03.453ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
1742026-01-30T21:13:03.467ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1752026-01-30T21:13:03.474ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1762026-01-30T21:13:03.478ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1772026-01-30T21:13:04.070ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1782026-01-30T21:13:04.078ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1792026-01-30T21:13:04.082ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1802026-01-30T21:13:04.277ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
1812026-01-30T21:13:05.552ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1822026-01-30T21:13:05.559ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1832026-01-30T21:13:05.564ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1842026-01-30T21:13:06.975ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1852026-01-30T21:13:06.983ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1862026-01-30T21:13:06.988ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1872026-01-30T21:13:08.164ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1882026-01-30T21:13:08.169ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1892026-01-30T21:13:08.240ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1902026-01-30T21:13:09.419ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1912026-01-30T21:13:09.424ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1922026-01-30T21:13:09.427ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1932026-01-30T21:13:10.680ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1942026-01-30T21:13:10.684ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1952026-01-30T21:13:10.687ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1962026-01-30T21:13:11.945ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1972026-01-30T21:13:11.951ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
1982026-01-30T21:13:11.954ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1992026-01-30T21:13:12.753ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2002026-01-30T21:13:12.773ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2012026-01-30T21:13:12.777ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2022026-01-30T21:13:13.456ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
2032026-01-30T21:13:13.592ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2042026-01-30T21:13:13.600ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2052026-01-30T21:13:13.607ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2062026-01-30T21:13:14.277ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
2072026-01-30T21:13:15.006ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2082026-01-30T21:13:15.012ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2092026-01-30T21:13:15.015ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2102026-01-30T21:13:15.921ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2112026-01-30T21:13:15.925ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2122026-01-30T21:13:15.927ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2132026-01-30T21:13:16.929ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2142026-01-30T21:13:16.933ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2152026-01-30T21:13:16.936ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2162026-01-30T21:13:17.310ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2172026-01-30T21:13:17.313ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 54.345961371s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
2182026-01-30T21:13:17.569ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2192026-01-30T21:13:17.574ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2202026-01-30T21:13:17.577ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2212026-01-30T21:13:18.375ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3cd0169a-e134-4091-812e-538f3f58ea17.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 706, tv_nsec: 385349560 } }) unit = producer-server
2222026-01-30T21:13:18.375ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
2232026-01-30T21:13:18.467ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
2242026-01-30T21:13:18.606ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2252026-01-30T21:13:18.611ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2262026-01-30T21:13:18.614ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2272026-01-30T21:13:19.517ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2282026-01-30T21:13:19.523ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2292026-01-30T21:13:19.527ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2302026-01-30T21:13:20.352ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2312026-01-30T21:13:20.356ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2322026-01-30T21:13:20.359ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2332026-01-30T21:13:21.582ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-01-30T21:13:21.587ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2352026-01-30T21:13:21.590ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2362026-01-30T21:13:23.058ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2372026-01-30T21:13:23.063ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2382026-01-30T21:13:23.067ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2392026-01-30T21:13:23.458ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
2402026-01-30T21:13:24.278ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
2412026-01-30T21:13:24.413ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2422026-01-30T21:13:24.420ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2432026-01-30T21:13:24.423ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2442026-01-30T21:13:25.016ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2452026-01-30T21:13:25.022ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2462026-01-30T21:13:25.025ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2472026-01-30T21:13:26.446ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2482026-01-30T21:13:26.452ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2492026-01-30T21:13:26.455ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2502026-01-30T21:13:27.572ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2512026-01-30T21:13:27.577ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2522026-01-30T21:13:27.580ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2532026-01-30T21:13:28.791ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2542026-01-30T21:13:28.799ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2552026-01-30T21:13:28.804ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2562026-01-30T21:13:30.284ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2572026-01-30T21:13:30.298ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2582026-01-30T21:13:30.342ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2592026-01-30T21:13:31.669ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2602026-01-30T21:13:31.683ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2612026-01-30T21:13:31.700ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2622026-01-30T21:13:32.655ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2632026-01-30T21:13:32.664ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2642026-01-30T21:13:32.673ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2652026-01-30T21:13:33.308ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2662026-01-30T21:13:33.315ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2672026-01-30T21:13:33.320ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2682026-01-30T21:13:33.581ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
2692026-01-30T21:13:33.590ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2702026-01-30T21:13:33.655ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 70.686570698s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
2712026-01-30T21:13:34.060ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722026-01-30T21:13:34.067ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2732026-01-30T21:13:34.071ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2742026-01-30T21:13:34.281ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
2752026-01-30T21:13:36.139ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2762026-01-30T21:13:36.148ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2772026-01-30T21:13:36.153ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2782026-01-30T21:13:37.067ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3cd0169a-e134-4091-812e-538f3f58ea17.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 725, tv_nsec: 75981009 } }) unit = producer-server
2792026-01-30T21:13:37.102ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
2802026-01-30T21:13:37.391ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
2812026-01-30T21:13:37.405ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2822026-01-30T21:13:37.413ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2832026-01-30T21:13:37.418ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2842026-01-30T21:13:38.764ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2852026-01-30T21:13:38.770ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2862026-01-30T21:13:38.774ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2872026-01-30T21:13:40.260ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2882026-01-30T21:13:40.267ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2892026-01-30T21:13:40.272ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2902026-01-30T21:13:40.991ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2912026-01-30T21:13:40.998ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2922026-01-30T21:13:41.002ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2932026-01-30T21:13:41.582ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2942026-01-30T21:13:41.590ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2952026-01-30T21:13:41.594ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2962026-01-30T21:13:42.214ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2972026-01-30T21:13:42.220ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
2982026-01-30T21:13:42.223ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2992026-01-30T21:13:43.592ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
3002026-01-30T21:13:43.659ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3012026-01-30T21:13:43.767ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3022026-01-30T21:13:43.770ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3032026-01-30T21:13:44.284ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
3042026-01-30T21:13:44.433ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3052026-01-30T21:13:44.439ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3062026-01-30T21:13:44.445ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3072026-01-30T21:13:45.213ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3082026-01-30T21:13:45.218ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3092026-01-30T21:13:45.221ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3102026-01-30T21:13:46.155ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3112026-01-30T21:13:46.160ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3122026-01-30T21:13:46.163ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3132026-01-30T21:13:47.159ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3142026-01-30T21:13:47.165ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3152026-01-30T21:13:47.318ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3162026-01-30T21:13:48.062ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3172026-01-30T21:13:48.067ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3182026-01-30T21:13:48.070ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3192026-01-30T21:13:48.728ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3202026-01-30T21:13:48.733ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3212026-01-30T21:13:48.736ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3222026-01-30T21:13:49.939ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3232026-01-30T21:13:49.945ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3242026-01-30T21:13:49.948ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3252026-01-30T21:13:50.559ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3262026-01-30T21:13:50.565ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3272026-01-30T21:13:50.568ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3282026-01-30T21:13:51.202ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3292026-01-30T21:13:51.211ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3302026-01-30T21:13:51.220ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3312026-01-30T21:13:52.393ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3322026-01-30T21:13:52.394ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 89.424543629s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
3332026-01-30T21:13:52.695ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3342026-01-30T21:13:52.701ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3352026-01-30T21:13:52.704ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3362026-01-30T21:13:53.625ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
3372026-01-30T21:13:53.844ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382026-01-30T21:13:53.849ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3392026-01-30T21:13:53.851ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3402026-01-30T21:13:54.286ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
3412026-01-30T21:13:55.041ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3422026-01-30T21:13:55.046ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3432026-01-30T21:13:55.049ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3442026-01-30T21:13:55.866ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3452026-01-30T21:13:55.873ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3462026-01-30T21:13:55.876ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3472026-01-30T21:13:56.513ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3482026-01-30T21:13:56.518ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3492026-01-30T21:13:56.521ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3502026-01-30T21:13:57.337ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3512026-01-30T21:13:57.342ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3522026-01-30T21:13:57.345ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3532026-01-30T21:13:58.100ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3cd0169a-e134-4091-812e-538f3f58ea17.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 746, tv_nsec: 109471497 } }) unit = producer-server
3542026-01-30T21:13:58.101ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
3552026-01-30T21:13:58.134ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
3562026-01-30T21:13:58.671ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3572026-01-30T21:13:58.684ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3582026-01-30T21:13:58.954ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3592026-01-30T21:13:59.728ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3602026-01-30T21:13:59.733ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3612026-01-30T21:13:59.736ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3622026-01-30T21:14:00.488ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3632026-01-30T21:14:00.493ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3642026-01-30T21:14:00.496ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3652026-01-30T21:14:01.838ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3662026-01-30T21:14:01.843ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3672026-01-30T21:14:01.846ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3682026-01-30T21:14:03.357ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3692026-01-30T21:14:03.364ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3702026-01-30T21:14:03.368ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3712026-01-30T21:14:03.628ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
3722026-01-30T21:14:04.289ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
3732026-01-30T21:14:04.762ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3742026-01-30T21:14:04.769ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3752026-01-30T21:14:04.773ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3762026-01-30T21:14:05.793ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3772026-01-30T21:14:05.800ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3782026-01-30T21:14:05.805ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3792026-01-30T21:14:06.437ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3802026-01-30T21:14:06.443ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3812026-01-30T21:14:06.446ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3822026-01-30T21:14:07.152ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3832026-01-30T21:14:07.158ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3842026-01-30T21:14:07.161ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3852026-01-30T21:14:08.126ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3862026-01-30T21:14:08.132ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3872026-01-30T21:14:08.136ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3882026-01-30T21:14:08.868ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3892026-01-30T21:14:08.875ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3902026-01-30T21:14:08.879ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3912026-01-30T21:14:09.925ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3922026-01-30T21:14:09.932ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3932026-01-30T21:14:09.936ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3942026-01-30T21:14:10.515ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3952026-01-30T21:14:10.520ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3962026-01-30T21:14:10.524ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3972026-01-30T21:14:11.344ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3982026-01-30T21:14:11.350ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
3992026-01-30T21:14:11.356ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4002026-01-30T21:14:12.334ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4012026-01-30T21:14:12.341ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4022026-01-30T21:14:12.345ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4032026-01-30T21:14:13.134ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4042026-01-30T21:14:13.135ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 110.167643697s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
4052026-01-30T21:14:13.631ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
4062026-01-30T21:14:13.711ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4072026-01-30T21:14:13.719ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4082026-01-30T21:14:13.722ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4092026-01-30T21:14:14.291ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
4102026-01-30T21:14:14.689ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4112026-01-30T21:14:14.695ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4122026-01-30T21:14:14.698ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4132026-01-30T21:14:15.472ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4142026-01-30T21:14:15.477ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4152026-01-30T21:14:15.480ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4162026-01-30T21:14:16.090ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4172026-01-30T21:14:16.096ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4182026-01-30T21:14:16.100ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4192026-01-30T21:14:17.267ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4202026-01-30T21:14:17.272ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4212026-01-30T21:14:17.275ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4222026-01-30T21:14:18.286ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4232026-01-30T21:14:18.291ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4242026-01-30T21:14:18.294ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4252026-01-30T21:14:18.907ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4262026-01-30T21:14:18.912ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4272026-01-30T21:14:18.914ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4282026-01-30T21:14:19.719ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4292026-01-30T21:14:19.725ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4302026-01-30T21:14:19.728ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4312026-01-30T21:14:20.534ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4322026-01-30T21:14:20.540ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4332026-01-30T21:14:20.543ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4342026-01-30T21:14:21.533ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4352026-01-30T21:14:21.540ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4362026-01-30T21:14:21.543ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4372026-01-30T21:14:22.301ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4382026-01-30T21:14:22.308ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4392026-01-30T21:14:22.311ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4402026-01-30T21:14:22.873ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4412026-01-30T21:14:22.879ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4422026-01-30T21:14:22.882ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4432026-01-30T21:14:23.632ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
4442026-01-30T21:14:24.099ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4452026-01-30T21:14:24.106ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4462026-01-30T21:14:24.109ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4472026-01-30T21:14:24.294ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
4482026-01-30T21:14:24.714ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3cd0169a-e134-4091-812e-538f3f58ea17.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 772, tv_nsec: 724421006 } }) unit = producer-server
4492026-01-30T21:14:24.715ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
4502026-01-30T21:14:24.775ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
4512026-01-30T21:14:24.998ZDEBGtfportd (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "9f3a3394-97c8-46e2-b16c-352c679971b2", "content-length": "133", "date": "Fri, 30 Jan 2026 21:14:24 GMT"} }) unit = producer-server
4522026-01-30T21:14:25.002ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 122.034537931s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"9f3a3394-97c8-46e2-b16c-352c679971b2\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 30 Jan 2026 21:14:24 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"9f3a3394-97c8-46e2-b16c-352c679971b2\\" }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/64b40cb/oximeter/producer/src/lib.rs:371 unit = producer-server
4532026-01-30T21:14:25.340ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4542026-01-30T21:14:25.348ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4552026-01-30T21:14:25.354ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4562026-01-30T21:14:26.695ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4572026-01-30T21:14:26.701ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4582026-01-30T21:14:26.705ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4592026-01-30T21:14:28.084ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4602026-01-30T21:14:28.090ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4612026-01-30T21:14:28.093ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4622026-01-30T21:14:29.213ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4632026-01-30T21:14:29.220ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4642026-01-30T21:14:29.223ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4652026-01-30T21:14:30.563ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4662026-01-30T21:14:30.569ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4672026-01-30T21:14:30.572ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4682026-01-30T21:14:31.430ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4692026-01-30T21:14:31.435ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4702026-01-30T21:14:31.439ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4712026-01-30T21:14:32.831ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4722026-01-30T21:14:32.837ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4732026-01-30T21:14:32.840ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4742026-01-30T21:14:33.635ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
4752026-01-30T21:14:33.962ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4762026-01-30T21:14:33.968ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4772026-01-30T21:14:33.971ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4782026-01-30T21:14:34.297ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
4792026-01-30T21:14:34.822ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4802026-01-30T21:14:34.828ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4812026-01-30T21:14:34.832ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4822026-01-30T21:14:35.472ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4832026-01-30T21:14:35.477ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4842026-01-30T21:14:35.480ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4852026-01-30T21:14:36.169ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4862026-01-30T21:14:36.174ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4872026-01-30T21:14:36.177ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4882026-01-30T21:14:37.113ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4892026-01-30T21:14:37.118ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4902026-01-30T21:14:37.122ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4912026-01-30T21:14:38.364ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4922026-01-30T21:14:38.370ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4932026-01-30T21:14:38.372ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4942026-01-30T21:14:39.121ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4952026-01-30T21:14:39.127ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4962026-01-30T21:14:39.131ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4972026-01-30T21:14:39.999ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4982026-01-30T21:14:40.004ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
4992026-01-30T21:14:40.007ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5002026-01-30T21:14:41.475ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5012026-01-30T21:14:41.480ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5022026-01-30T21:14:41.484ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5032026-01-30T21:14:42.973ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5042026-01-30T21:14:42.977ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5052026-01-30T21:14:42.979ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5062026-01-30T21:14:43.618ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("3cd0169a-e134-4091-812e-538f3f58ea17.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c5cf11ed-b0bd-43e5-ba22-32fbc366267d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 791, tv_nsec: 628408013 } }) unit = producer-server
5072026-01-30T21:14:43.619ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
5082026-01-30T21:14:43.651ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
5092026-01-30T21:14:43.652ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
5102026-01-30T21:14:43.703ZDEBGtfportd (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "7d18f718-e6f6-447d-9686-e857173c82c6", "content-length": "41", "date": "Fri, 30 Jan 2026 21:14:43 GMT"} }) unit = producer-server
5112026-01-30T21:14:43.704ZDEBGtfportd (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s unit = producer-server wait_period = 150s
5122026-01-30T21:14:43.766ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5132026-01-30T21:14:43.771ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5142026-01-30T21:14:43.774ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5152026-01-30T21:14:44.298ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
5162026-01-30T21:14:44.650ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5172026-01-30T21:14:44.655ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5182026-01-30T21:14:44.658ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5192026-01-30T21:14:45.238ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5202026-01-30T21:14:45.243ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5212026-01-30T21:14:45.247ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5222026-01-30T21:14:45.881ZINFOtfportd (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::2]:37579 remote_addr = [fd00:1122:3344:101::d]:52673 unit = producer-server
5232026-01-30T21:14:45.893ZINFOtfportd (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 8458 local_addr = [fd00:1122:3344:101::2]:37579 method = GET remote_addr = [fd00:1122:3344:101::d]:52673 req_id = 5c002c92-064b-46a6-9f4d-bc57fc1995d0 response_code = 200 unit = producer-server uri = /5b03a63f-53bc-4eba-a700-5c0784456ec7
5242026-01-30T21:14:46.058ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5252026-01-30T21:14:46.063ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5262026-01-30T21:14:46.066ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5272026-01-30T21:14:46.594ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5282026-01-30T21:14:46.599ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5292026-01-30T21:14:46.603ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5302026-01-30T21:14:47.856ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5312026-01-30T21:14:47.862ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5322026-01-30T21:14:47.865ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5332026-01-30T21:14:48.741ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5342026-01-30T21:14:48.746ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5352026-01-30T21:14:48.749ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5362026-01-30T21:14:49.996ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5372026-01-30T21:14:50.001ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5382026-01-30T21:14:50.005ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5392026-01-30T21:14:50.931ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5402026-01-30T21:14:50.937ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5412026-01-30T21:14:50.941ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5422026-01-30T21:14:51.566ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5432026-01-30T21:14:51.572ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5442026-01-30T21:14:51.575ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5452026-01-30T21:14:52.849ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5462026-01-30T21:14:52.855ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5472026-01-30T21:14:52.859ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5482026-01-30T21:14:53.655ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11886258443777427216 n_samples = 6 unit = oximeter
5492026-01-30T21:14:54.301ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 972073342625437383 n_samples = 6 unit = oximeter
5502026-01-30T21:14:54.313ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5512026-01-30T21:14:54.319ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5522026-01-30T21:14:54.323ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5532026-01-30T21:14:55.531ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5542026-01-30T21:14:55.538ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5552026-01-30T21:14:55.542ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5562026-01-30T21:14:56.455ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5572026-01-30T21:14:56.461ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5582026-01-30T21:14:56.465ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5592026-01-30T21:14:57.558ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5602026-01-30T21:14:57.564ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5612026-01-30T21:14:57.568ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5622026-01-30T21:14:58.694ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5632026-01-30T21:14:58.700ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5642026-01-30T21:14:58.704ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5652026-01-30T21:14:59.470ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5662026-01-30T21:14:59.476ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5672026-01-30T21:14:59.479ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5682026-01-30T21:15:00.340ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5692026-01-30T21:15:00.346ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::bcbb:3fff:fe6e:41e7]
5702026-01-30T21:15:00.350ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]