Max level shown:
1 [ Jun 10 20:57:25 Enabled. ]
2 [ Jun 10 20:57:25 Rereading configuration. ]
3 [ Jun 10 20:57:26 Rereading configuration. ]
4 [ Jun 10 20:57:37 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jun 10 20:57:37 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-06-10T20:57:37.733ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-06-10T20:57:37.735ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
92026-06-10T20:57:37.735ZINFOtfportd: running in sync-only mode
102026-06-10T20:57:37.737ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-06-10T20:57:37.737ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-06-10T20:57:37.739ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 4be2375e-273a-41c6-b757-a7da9d100759, ), sled_id: Some( f2863d7c-ea5c-431e-9453-d23c733de63b, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
142026-06-10T20:57:37.739ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-06-10T20:57:37.744ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::4829:fff:feaf:8a69->4a:29:0f:af:8a:69 interface = tfportrear0_0
162026-06-10T20:57:37.745ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-06-10T20:57:37.750ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
182026-06-10T20:57:37.755ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Jun 10 20:57:38 Rereading configuration. ]
20 [ Jun 10 20:57:38 Executing refresh method (:kill). ]
21 [ Jun 10 20:57:38 Stopping because all processes in service exited. ]
22 [ Jun 10 20:57:38 Executing stop method (:kill). ]
23 [ Jun 10 20:57:38 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Jun 10 20:57:38 Method "start" exited with status 0. ]
252026-06-10T20:57:38.367ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-06-10T20:57:38.368ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
272026-06-10T20:57:38.368ZINFOtfportd: running in sync-only mode
282026-06-10T20:57:38.368ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-06-10T20:57:38.370ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-06-10T20:57:38.370ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 4be2375e-273a-41c6-b757-a7da9d100759, sled_id: f2863d7c-ea5c-431e-9453-d23c733de63b, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: b8cf297e-4f78-4377-8b81-c5127728e624, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-06-10T20:57:38.370ZDEBGtfportd: created producer ID producer_id = 01ab235c-821e-41af-a885-b0f499050746 unit = oximeter
322026-06-10T20:57:38.371ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 4be2375e-273a-41c6-b757-a7da9d100759, ), sled_id: Some( f2863d7c-ea5c-431e-9453-d23c733de63b, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
332026-06-10T20:57:38.379ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-06-10T20:57:38.381ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::2010:abff:fe05:da3->22:10:ab:05:0d:a3 interface = tfportrear0_0
352026-06-10T20:57:38.381ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::4829:fff:feaf:8a69->4a:29:0f:af:8a:69 interface = tfportrear0_0
362026-06-10T20:57:38.383ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-06-10T20:57:38.384ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:63840 method = GET path = /{producer_id} unit = producer-server versions = all
382026-06-10T20:57:38.384ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:63840 unit = producer-server
392026-06-10T20:57:38.384ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:63840 unit = producer-server
402026-06-10T20:57:38.384ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:63840 unit = producer-server
412026-06-10T20:57:38.384ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-06-10T20:57:38.384ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-06-10T20:57:38.384ZDEBGtfportd: starting producer registration task unit = producer-server
442026-06-10T20:57:38.384ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:63840 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 01ab235c-821e-41af-a885-b0f499050746 unit = producer-server
452026-06-10T20:57:38.384ZINFOtfportd: created oximeter producer server unit = oximeter
462026-06-10T20:57:38.384ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-06-10T20:57:38.384ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
482026-06-10T20:57:38.385ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-06-10T20:57:38.386ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 1.120448ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
502026-06-10T20:57:38.386ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-06-10T20:57:21.452002850Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
512026-06-10T20:57:38.386ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 17472123414372464905 unit = oximeter
522026-06-10T20:57:38.386ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
532026-06-10T20:57:38.388ZWARNtfportd: tfportqsfp0_0 has no addrs
542026-06-10T20:57:38.654ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("300e4575-e36a-48da-9cad-1576a4829c84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("477d5d28-bd17-45cb-abef-f249bc98911a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 725, tv_nsec: 663061179 } }) unit = producer-server
552026-06-10T20:57:38.654ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
562026-06-10T20:57:38.670ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
572026-06-10T20:57:38.974ZINFOtfportd: added new ARP entry entry = 10.151.2.174->8e:07:51:2f:4e:8d interface = tfportqsfp0_0
582026-06-10T20:57:39.796ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
592026-06-10T20:57:39.803ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
602026-06-10T20:57:39.806ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
612026-06-10T20:57:39.807ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
622026-06-10T20:57:39.808ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-06-10T20:57:21.451869366Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
632026-06-10T20:57:39.808ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 11610904761549124466 unit = oximeter
642026-06-10T20:57:39.808ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
652026-06-10T20:57:41.019ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
662026-06-10T20:57:41.025ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
672026-06-10T20:57:41.028ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
682026-06-10T20:57:42.101ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
692026-06-10T20:57:42.105ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
702026-06-10T20:57:42.108ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
712026-06-10T20:57:42.954ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
722026-06-10T20:57:42.960ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
732026-06-10T20:57:42.963ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
742026-06-10T20:57:44.289ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
752026-06-10T20:57:44.293ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
762026-06-10T20:57:44.296ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
772026-06-10T20:57:45.122ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
782026-06-10T20:57:45.126ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
792026-06-10T20:57:45.130ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
802026-06-10T20:57:45.969ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
812026-06-10T20:57:45.973ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
822026-06-10T20:57:45.976ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
832026-06-10T20:57:46.564ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
842026-06-10T20:57:46.569ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
852026-06-10T20:57:46.572ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
862026-06-10T20:57:47.423ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
872026-06-10T20:57:47.427ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
882026-06-10T20:57:47.430ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
892026-06-10T20:57:48.277ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
902026-06-10T20:57:48.282ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
912026-06-10T20:57:48.285ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
922026-06-10T20:57:48.389ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
932026-06-10T20:57:48.840ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
942026-06-10T20:57:48.845ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
952026-06-10T20:57:48.848ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
962026-06-10T20:57:49.810ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
972026-06-10T20:57:50.175ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
982026-06-10T20:57:50.179ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
992026-06-10T20:57:50.182ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1002026-06-10T20:57:51.560ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1012026-06-10T20:57:51.565ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1022026-06-10T20:57:51.568ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1032026-06-10T20:57:53.288ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1042026-06-10T20:57:53.293ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1052026-06-10T20:57:53.296ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1062026-06-10T20:57:53.839ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1072026-06-10T20:57:53.843ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1082026-06-10T20:57:53.846ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1092026-06-10T20:57:54.224ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1102026-06-10T20:57:54.225ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 15.288518932s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1112026-06-10T20:57:54.565ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1122026-06-10T20:57:54.570ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1132026-06-10T20:57:54.574ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1142026-06-10T20:57:54.825ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("300e4575-e36a-48da-9cad-1576a4829c84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("477d5d28-bd17-45cb-abef-f249bc98911a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 741, tv_nsec: 283363975 } }) unit = producer-server
1152026-06-10T20:57:54.825ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
1162026-06-10T20:57:54.833ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1172026-06-10T20:57:56.038ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1182026-06-10T20:57:56.044ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1192026-06-10T20:57:56.047ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1202026-06-10T20:57:57.513ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1212026-06-10T20:57:57.518ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1222026-06-10T20:57:57.522ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1232026-06-10T20:57:58.734ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1242026-06-10T20:57:58.738ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1252026-06-10T20:57:58.742ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1262026-06-10T20:57:58.930ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
1272026-06-10T20:58:00.133ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1282026-06-10T20:58:00.137ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1292026-06-10T20:58:00.140ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1302026-06-10T20:58:00.346ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
1312026-06-10T20:58:01.363ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1322026-06-10T20:58:01.369ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1332026-06-10T20:58:01.372ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1342026-06-10T20:58:02.544ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1352026-06-10T20:58:02.549ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1362026-06-10T20:58:02.553ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1372026-06-10T20:58:03.107ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1382026-06-10T20:58:03.111ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1392026-06-10T20:58:03.115ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1402026-06-10T20:58:03.899ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1412026-06-10T20:58:03.905ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1422026-06-10T20:58:03.908ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1432026-06-10T20:58:04.977ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1442026-06-10T20:58:04.982ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1452026-06-10T20:58:04.986ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1462026-06-10T20:58:05.941ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1472026-06-10T20:58:05.946ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1482026-06-10T20:58:05.950ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1492026-06-10T20:58:06.492ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1502026-06-10T20:58:06.497ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1512026-06-10T20:58:06.500ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1522026-06-10T20:58:07.992ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1532026-06-10T20:58:07.997ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1542026-06-10T20:58:08.000ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1552026-06-10T20:58:08.903ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
1562026-06-10T20:58:09.187ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1572026-06-10T20:58:09.191ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1582026-06-10T20:58:09.194ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1592026-06-10T20:58:09.791ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1602026-06-10T20:58:09.793ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 30.899740729s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1612026-06-10T20:58:10.320ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
1622026-06-10T20:58:10.504ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("300e4575-e36a-48da-9cad-1576a4829c84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("477d5d28-bd17-45cb-abef-f249bc98911a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 757, tv_nsec: 6690879 } }) unit = producer-server
1632026-06-10T20:58:10.505ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
1642026-06-10T20:58:10.513ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1652026-06-10T20:58:10.596ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1662026-06-10T20:58:10.600ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1672026-06-10T20:58:10.603ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1682026-06-10T20:58:11.784ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1692026-06-10T20:58:11.789ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1702026-06-10T20:58:11.791ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1712026-06-10T20:58:12.902ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1722026-06-10T20:58:12.906ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1732026-06-10T20:58:12.910ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1742026-06-10T20:58:14.080ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1752026-06-10T20:58:14.085ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1762026-06-10T20:58:14.087ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1772026-06-10T20:58:14.724ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1782026-06-10T20:58:14.729ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1792026-06-10T20:58:14.732ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1802026-06-10T20:58:15.929ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1812026-06-10T20:58:15.934ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1822026-06-10T20:58:15.938ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1832026-06-10T20:58:17.388ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1842026-06-10T20:58:17.394ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1852026-06-10T20:58:17.397ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1862026-06-10T20:58:18.138ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1872026-06-10T20:58:18.143ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1882026-06-10T20:58:18.146ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1892026-06-10T20:58:18.680ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1902026-06-10T20:58:18.685ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1912026-06-10T20:58:18.688ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1922026-06-10T20:58:18.879ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
1932026-06-10T20:58:19.635ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1942026-06-10T20:58:19.641ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1952026-06-10T20:58:19.644ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1962026-06-10T20:58:20.295ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
1972026-06-10T20:58:21.114ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1982026-06-10T20:58:21.119ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
1992026-06-10T20:58:21.122ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2002026-06-10T20:58:22.607ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2012026-06-10T20:58:22.612ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2022026-06-10T20:58:22.615ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2032026-06-10T20:58:23.920ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2042026-06-10T20:58:23.924ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2052026-06-10T20:58:23.927ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2062026-06-10T20:58:24.719ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2072026-06-10T20:58:24.724ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2082026-06-10T20:58:24.728ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2092026-06-10T20:58:25.473ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2102026-06-10T20:58:25.473ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 46.623453006s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2112026-06-10T20:58:25.842ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2122026-06-10T20:58:25.846ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2132026-06-10T20:58:25.849ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2142026-06-10T20:58:26.493ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2152026-06-10T20:58:26.498ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2162026-06-10T20:58:26.501ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2172026-06-10T20:58:26.628ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("300e4575-e36a-48da-9cad-1576a4829c84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("477d5d28-bd17-45cb-abef-f249bc98911a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 773, tv_nsec: 174716681 } }) unit = producer-server
2182026-06-10T20:58:26.629ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
2192026-06-10T20:58:26.637ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
2202026-06-10T20:58:27.035ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2212026-06-10T20:58:27.039ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2222026-06-10T20:58:27.042ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2232026-06-10T20:58:28.216ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2242026-06-10T20:58:28.220ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2252026-06-10T20:58:28.222ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2262026-06-10T20:58:28.853ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
2272026-06-10T20:58:29.396ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2282026-06-10T20:58:29.405ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2292026-06-10T20:58:29.408ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2302026-06-10T20:58:29.988ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2312026-06-10T20:58:29.993ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2322026-06-10T20:58:29.996ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2332026-06-10T20:58:30.269ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
2342026-06-10T20:58:31.219ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2352026-06-10T20:58:31.224ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2362026-06-10T20:58:31.227ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2372026-06-10T20:58:32.469ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2382026-06-10T20:58:32.474ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2392026-06-10T20:58:32.477ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2402026-06-10T20:58:33.655ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2412026-06-10T20:58:33.659ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2422026-06-10T20:58:33.663ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2432026-06-10T20:58:34.880ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2442026-06-10T20:58:34.885ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2452026-06-10T20:58:34.887ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2462026-06-10T20:58:35.951ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2472026-06-10T20:58:35.956ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2482026-06-10T20:58:35.959ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2492026-06-10T20:58:36.799ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2502026-06-10T20:58:36.804ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2512026-06-10T20:58:36.808ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2522026-06-10T20:58:38.279ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2532026-06-10T20:58:38.283ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2542026-06-10T20:58:38.286ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2552026-06-10T20:58:38.828ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
2562026-06-10T20:58:39.204ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2572026-06-10T20:58:39.208ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2582026-06-10T20:58:39.211ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2592026-06-10T20:58:40.244ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
2602026-06-10T20:58:40.555ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2612026-06-10T20:58:40.559ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2622026-06-10T20:58:40.562ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2632026-06-10T20:58:41.252ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2642026-06-10T20:58:41.256ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2652026-06-10T20:58:41.260ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2662026-06-10T20:58:41.596ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2672026-06-10T20:58:41.598ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 62.791639894s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2682026-06-10T20:58:41.804ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692026-06-10T20:58:41.809ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2702026-06-10T20:58:41.812ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2712026-06-10T20:58:43.027ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722026-06-10T20:58:43.032ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2732026-06-10T20:58:43.035ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2742026-06-10T20:58:44.142ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2752026-06-10T20:58:44.147ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2762026-06-10T20:58:44.150ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2772026-06-10T20:58:44.823ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2782026-06-10T20:58:44.828ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2792026-06-10T20:58:44.831ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2802026-06-10T20:58:45.352ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("300e4575-e36a-48da-9cad-1576a4829c84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("477d5d28-bd17-45cb-abef-f249bc98911a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 791, tv_nsec: 949882135 } }) unit = producer-server
2812026-06-10T20:58:45.352ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
2822026-06-10T20:58:45.359ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
2832026-06-10T20:58:46.230ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2842026-06-10T20:58:46.235ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2852026-06-10T20:58:46.237ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2862026-06-10T20:58:47.290ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2872026-06-10T20:58:47.295ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2882026-06-10T20:58:47.298ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2892026-06-10T20:58:48.169ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2902026-06-10T20:58:48.174ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2912026-06-10T20:58:48.177ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2922026-06-10T20:58:48.803ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
2932026-06-10T20:58:49.320ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2942026-06-10T20:58:49.325ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2952026-06-10T20:58:49.328ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2962026-06-10T20:58:49.870ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2972026-06-10T20:58:49.874ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
2982026-06-10T20:58:49.877ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2992026-06-10T20:58:50.219ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
3002026-06-10T20:58:51.274ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3012026-06-10T20:58:51.279ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3022026-06-10T20:58:51.282ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3032026-06-10T20:58:51.992ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3042026-06-10T20:58:51.997ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3052026-06-10T20:58:52.000ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3062026-06-10T20:58:52.597ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3072026-06-10T20:58:52.601ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3082026-06-10T20:58:52.605ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3092026-06-10T20:58:53.416ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3102026-06-10T20:58:53.420ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3112026-06-10T20:58:53.422ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3122026-06-10T20:58:53.938ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3132026-06-10T20:58:53.943ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3142026-06-10T20:58:53.946ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3152026-06-10T20:58:54.544ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3162026-06-10T20:58:54.549ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3172026-06-10T20:58:54.552ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3182026-06-10T20:58:55.072ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3192026-06-10T20:58:55.077ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3202026-06-10T20:58:55.080ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3212026-06-10T20:58:55.867ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3222026-06-10T20:58:55.873ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3232026-06-10T20:58:55.876ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3242026-06-10T20:58:57.346ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3252026-06-10T20:58:57.352ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3262026-06-10T20:58:57.354ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3272026-06-10T20:58:58.750ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3282026-06-10T20:58:58.755ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3292026-06-10T20:58:58.758ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3302026-06-10T20:58:58.777ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
3312026-06-10T20:58:59.452ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3322026-06-10T20:58:59.457ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3332026-06-10T20:58:59.460ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3342026-06-10T20:59:00.193ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
3352026-06-10T20:59:00.319ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3362026-06-10T20:59:00.321ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 81.565813845s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3372026-06-10T20:59:00.436ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382026-06-10T20:59:00.440ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3392026-06-10T20:59:00.443ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3402026-06-10T20:59:01.224ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3412026-06-10T20:59:01.229ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3422026-06-10T20:59:01.232ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3432026-06-10T20:59:02.217ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3442026-06-10T20:59:02.222ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3452026-06-10T20:59:02.224ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3462026-06-10T20:59:02.819ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3472026-06-10T20:59:02.824ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3482026-06-10T20:59:02.827ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3492026-06-10T20:59:04.024ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3502026-06-10T20:59:04.029ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3512026-06-10T20:59:04.032ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3522026-06-10T20:59:04.627ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3532026-06-10T20:59:04.632ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3542026-06-10T20:59:04.635ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3552026-06-10T20:59:05.588ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3562026-06-10T20:59:05.593ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3572026-06-10T20:59:05.596ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3582026-06-10T20:59:06.569ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3592026-06-10T20:59:06.574ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3602026-06-10T20:59:06.577ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3612026-06-10T20:59:06.608ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("300e4575-e36a-48da-9cad-1576a4829c84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("477d5d28-bd17-45cb-abef-f249bc98911a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 813, tv_nsec: 263909380 } }) unit = producer-server
3622026-06-10T20:59:06.609ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
3632026-06-10T20:59:06.616ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
3642026-06-10T20:59:07.168ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3652026-06-10T20:59:07.172ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3662026-06-10T20:59:07.175ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3672026-06-10T20:59:08.638ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3682026-06-10T20:59:08.643ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3692026-06-10T20:59:08.646ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3702026-06-10T20:59:08.753ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
3712026-06-10T20:59:09.714ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3722026-06-10T20:59:09.719ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3732026-06-10T20:59:09.722ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3742026-06-10T20:59:10.169ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
3752026-06-10T20:59:11.104ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3762026-06-10T20:59:11.109ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3772026-06-10T20:59:11.112ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3782026-06-10T20:59:12.379ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3792026-06-10T20:59:12.383ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3802026-06-10T20:59:12.386ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3812026-06-10T20:59:13.286ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3822026-06-10T20:59:13.291ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3832026-06-10T20:59:13.294ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3842026-06-10T20:59:14.734ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3852026-06-10T20:59:14.739ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3862026-06-10T20:59:14.742ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3872026-06-10T20:59:16.116ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3882026-06-10T20:59:16.120ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3892026-06-10T20:59:16.124ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3902026-06-10T20:59:17.442ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3912026-06-10T20:59:17.447ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3922026-06-10T20:59:17.449ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3932026-06-10T20:59:18.257ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3942026-06-10T20:59:18.261ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3952026-06-10T20:59:18.264ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3962026-06-10T20:59:18.727ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
3972026-06-10T20:59:18.780ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3982026-06-10T20:59:18.785ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
3992026-06-10T20:59:18.788ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4002026-06-10T20:59:20.081ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4012026-06-10T20:59:20.085ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4022026-06-10T20:59:20.088ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4032026-06-10T20:59:20.144ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
4042026-06-10T20:59:21.565ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4052026-06-10T20:59:21.569ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4062026-06-10T20:59:21.573ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4072026-06-10T20:59:21.575ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4082026-06-10T20:59:21.576ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 102.879353093s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
4092026-06-10T20:59:22.477ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4102026-06-10T20:59:22.481ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4112026-06-10T20:59:22.484ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4122026-06-10T20:59:23.037ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4132026-06-10T20:59:23.041ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4142026-06-10T20:59:23.044ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4152026-06-10T20:59:23.729ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4162026-06-10T20:59:23.734ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4172026-06-10T20:59:23.738ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4182026-06-10T20:59:24.990ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4192026-06-10T20:59:24.994ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4202026-06-10T20:59:24.997ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4212026-06-10T20:59:26.039ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4222026-06-10T20:59:26.044ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4232026-06-10T20:59:26.047ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4242026-06-10T20:59:26.890ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4252026-06-10T20:59:26.895ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4262026-06-10T20:59:26.897ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4272026-06-10T20:59:27.997ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4282026-06-10T20:59:28.001ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4292026-06-10T20:59:28.005ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4302026-06-10T20:59:28.702ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
4312026-06-10T20:59:29.162ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4322026-06-10T20:59:29.166ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4332026-06-10T20:59:29.170ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4342026-06-10T20:59:30.118ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
4352026-06-10T20:59:30.316ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4362026-06-10T20:59:30.320ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4372026-06-10T20:59:30.323ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4382026-06-10T20:59:30.687ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("300e4575-e36a-48da-9cad-1576a4829c84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("477d5d28-bd17-45cb-abef-f249bc98911a.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d76fe2c7-39b7-435b-a9d5-c0055ba98799.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 837, tv_nsec: 409200885 } }) unit = producer-server
4392026-06-10T20:59:30.688ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
4402026-06-10T20:59:30.695ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
4412026-06-10T20:59:31.026ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4422026-06-10T20:59:31.031ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4432026-06-10T20:59:31.034ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4442026-06-10T20:59:32.384ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4452026-06-10T20:59:32.389ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4462026-06-10T20:59:32.391ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4472026-06-10T20:59:33.267ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4482026-06-10T20:59:33.272ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4492026-06-10T20:59:33.275ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4502026-06-10T20:59:34.717ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4512026-06-10T20:59:34.722ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4522026-06-10T20:59:34.726ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4532026-06-10T20:59:35.943ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4542026-06-10T20:59:35.948ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4552026-06-10T20:59:35.952ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4562026-06-10T20:59:37.424ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4572026-06-10T20:59:37.428ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4582026-06-10T20:59:37.431ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4592026-06-10T20:59:37.951ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4602026-06-10T20:59:37.957ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4612026-06-10T20:59:37.960ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4622026-06-10T20:59:38.677ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
4632026-06-10T20:59:38.970ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4642026-06-10T20:59:38.975ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4652026-06-10T20:59:38.978ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4662026-06-10T20:59:39.831ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4672026-06-10T20:59:39.837ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4682026-06-10T20:59:39.840ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4692026-06-10T20:59:40.092ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
4702026-06-10T20:59:40.769ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4712026-06-10T20:59:40.773ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4722026-06-10T20:59:40.775ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4732026-06-10T20:59:42.168ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4742026-06-10T20:59:42.173ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4752026-06-10T20:59:42.177ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4762026-06-10T20:59:43.356ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4772026-06-10T20:59:43.360ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4782026-06-10T20:59:43.363ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4792026-06-10T20:59:44.761ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4802026-06-10T20:59:44.765ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4812026-06-10T20:59:44.768ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4822026-06-10T20:59:45.655ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
4832026-06-10T20:59:45.656ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 7 delay = 127.025025731s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
4842026-06-10T20:59:45.956ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4852026-06-10T20:59:45.961ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4862026-06-10T20:59:45.964ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4872026-06-10T20:59:46.808ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4882026-06-10T20:59:46.813ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4892026-06-10T20:59:46.816ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4902026-06-10T20:59:47.822ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4912026-06-10T20:59:47.826ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4922026-06-10T20:59:47.829ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4932026-06-10T20:59:48.651ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
4942026-06-10T20:59:48.657ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4952026-06-10T20:59:48.661ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4962026-06-10T20:59:48.664ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
4972026-06-10T20:59:49.954ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
4982026-06-10T20:59:49.959ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
4992026-06-10T20:59:49.962ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5002026-06-10T20:59:50.067ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
5012026-06-10T20:59:50.583ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5022026-06-10T20:59:50.588ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
5032026-06-10T20:59:50.591ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5042026-06-10T20:59:51.498ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5052026-06-10T20:59:51.503ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
5062026-06-10T20:59:51.506ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5072026-06-10T20:59:52.712ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5082026-06-10T20:59:52.717ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
5092026-06-10T20:59:52.719ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5102026-06-10T20:59:54.104ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5112026-06-10T20:59:54.109ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
5122026-06-10T20:59:54.112ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5132026-06-10T20:59:55.130ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5142026-06-10T20:59:55.134ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
5152026-06-10T20:59:55.138ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5162026-06-10T20:59:56.621ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5172026-06-10T20:59:56.626ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
5182026-06-10T20:59:56.629ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5192026-06-10T20:59:57.164ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5202026-06-10T20:59:57.169ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
5212026-06-10T20:59:57.172ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5222026-06-10T20:59:58.186ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5232026-06-10T20:59:58.191ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
5242026-06-10T20:59:58.194ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5252026-06-10T20:59:58.625ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 17472123414372464905 n_samples = 6 unit = oximeter
5262026-06-10T20:59:59.573ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5272026-06-10T20:59:59.577ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
5282026-06-10T20:59:59.580ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
5292026-06-10T21:00:00.043ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 11610904761549124466 n_samples = 6 unit = oximeter
5302026-06-10T21:00:00.685ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
5312026-06-10T21:00:00.691ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4829:fff:feaf:8a69]
5322026-06-10T21:00:00.695ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]