Max level shown:
1 [ Jul 2 06:12:55 Enabled. ]
2 [ Jul 2 06:12:55 Rereading configuration. ]
3 [ Jul 2 06:12:55 Rereading configuration. ]
4 [ Jul 2 06:13:06 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
5 [ Jul 2 06:13:06 Method "start" exited with status 0. ]
6 No non-localhost IPv6 addresses found in SMF properties
72026-07-02T06:13:06.643ZINFOtfportd: connecting to dpd host = [::1] port = 12224
82026-07-02T06:13:06.644ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
92026-07-02T06:13:06.645ZINFOtfportd: running in sync-only mode
102026-07-02T06:13:06.647ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
112026-07-02T06:13:06.647ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
12 No non-localhost IPv6 addresses found in SMF properties
132026-07-02T06:13:06.648ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 63b4f307-f92b-441d-97f6-ddb6aa1ac5c3, ), sled_id: Some( a322738b-2ac0-43ed-89a2-ad085640624d, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
142026-07-02T06:13:06.649ZINFOtfportd: no non-localhost IPv6 listen address available, waiting for SMF update unit = oximeter
152026-07-02T06:13:06.652ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::4c44:5cff:fe14:44a3->4e:44:5c:14:44:a3 interface = tfportrear0_0
162026-07-02T06:13:06.655ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
172026-07-02T06:13:06.661ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
182026-07-02T06:13:06.666ZWARNtfportd: tfportqsfp0_0 has no addrs
19 [ Jul 2 06:13:07 Rereading configuration. ]
20 [ Jul 2 06:13:07 Executing refresh method (:kill). ]
21 [ Jul 2 06:13:07 Stopping because all processes in service exited. ]
22 [ Jul 2 06:13:07 Executing stop method (:kill). ]
23 [ Jul 2 06:13:07 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/dendrite/bin/tfportd --sync-only &"). ]
24 [ Jul 2 06:13:07 Method "start" exited with status 0. ]
252026-07-02T06:13:07.285ZINFOtfportd: connecting to dpd host = [::1] port = 12224
262026-07-02T06:13:07.286ZINFOtfportd: connected to dpd running softnpu-cc0c307c-dirty
272026-07-02T06:13:07.287ZINFOtfportd: running in sync-only mode
282026-07-02T06:13:07.287ZINFOtfportd: found suitable IPv6 address for oximeter producer server address = fd00:1122:3344:101::2 unit = oximeter
292026-07-02T06:13:07.289ZINFOtfportd: refreshing SMF config prior to waiting on signals unit = signal_handler
302026-07-02T06:13:07.289ZDEBGtfportd: extracted oximeter configuration data config = OximeterConfig { listen_address: fd00:1122:3344:101::2, sled_identifiers: SledIdentifiers { rack_id: 63b4f307-f92b-441d-97f6-ddb6aa1ac5c3, sled_id: a322738b-2ac0-43ed-89a2-ad085640624d, model: "i86pc", revision: 0, serial: "sock" }, switch_identifiers: SwitchIdentifiers { asic_backend: "softnpu", fab: None, fuse: Some(FuseData { chip_rev: ChipRevision { device_id: 272, rev: "B1", rev_num: 2 }, disabled: DisabledFeatures { bsync: false, mau: [0, 0, 0, 0], pgen: false, pipes: 0, ports: 0, resub: false, speeds: 0, tm_mem: 0 }, frequency: FrequencySettings { bps: 3, bps_ext: 0, cpu_speed_dis: 0, disabled: false, pcie_dis: 0, pps: 3, pps_ext: 0 }, manufacturing: ManufacturingData { core_repair: false, die_rotation: false, pmro_and_skew: 0, silent_spin: 0, soft_pipe_dis: 0, tile_repair: false, voltage_scaling: 0, wf_core_repair: false }, part: PartInfo { part_num: 4660, pkg_id: 1, version: 2 } }), lot: None, lotnum: None, model: "FAKE_SIM_SIDECAR", revision: 0, serial: "SimSidecar0", sidecar_id: 6f097e66-7569-47e4-90e2-9e44a94ef6b6, slot: 0, wafer: None, wafer_loc: None } } unit = oximeter
312026-07-02T06:13:07.289ZDEBGtfportd: created producer ID producer_id = af9ffeea-a682-4014-8eb6-eefa55145138 unit = oximeter
322026-07-02T06:13:07.290ZINFOtfportd: refreshed config: Config { log_file: None, log_format: Json, listen_addresses: [ [fd00:1122:3344:101::2]:12231, [::1]:12231, ], pkt_source: None, dpd_host: "[::1]", dpd_port: 12224, vlan_link: None, vlan_data: None, techport0_prefix: None, techport1_prefix: None, rack_id: Some( 63b4f307-f92b-441d-97f6-ddb6aa1ac5c3, ), sled_id: Some( a322738b-2ac0-43ed-89a2-ad085640624d, ), sled_model: Some( "i86pc", ), sled_revision: Some( 0, ), sled_serial: Some( "sock", ), }
332026-07-02T06:13:07.298ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
342026-07-02T06:13:07.300ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::5848:fcff:fe62:ec9e->5a:48:fc:62:ec:9e interface = tfportrear0_0
352026-07-02T06:13:07.300ZINFOtfportd: added new NDP entry entry = tfportrear0_0:fe80::4c44:5cff:fe14:44a3->4e:44:5c:14:44:a3 interface = tfportrear0_0
362026-07-02T06:13:07.301ZDEBGtfportd: registered DTrace probes unit = producer-server
372026-07-02T06:13:07.303ZDEBGtfportd (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::2]:61676 method = GET path = /{producer_id} unit = producer-server versions = all
382026-07-02T06:13:07.303ZINFOtfportd (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::2]:61676 unit = producer-server
392026-07-02T06:13:07.303ZDEBGtfportd (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::2]:61676 unit = producer-server
402026-07-02T06:13:07.303ZDEBGtfportd: Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::2]:61676 unit = producer-server
412026-07-02T06:13:07.303ZDEBGtfportd: Nexus IP not provided, will use DNS to resolve it unit = producer-server
422026-07-02T06:13:07.303ZINFOtfportd (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/internal-dns/resolver/src/resolver.rs:144 unit = producer-server
432026-07-02T06:13:07.303ZDEBGtfportd: starting producer registration task unit = producer-server
442026-07-02T06:13:07.303ZINFOtfportd: starting oximeter metric producer server address = [fd00:1122:3344:101::2]:61676 file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:279 interval = 30s producer_id = af9ffeea-a682-4014-8eb6-eefa55145138 unit = producer-server
452026-07-02T06:13:07.303ZINFOtfportd: created oximeter producer server unit = oximeter
462026-07-02T06:13:07.303ZDEBGtfportd (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0 unit = producer-server
472026-07-02T06:13:07.304ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
482026-07-02T06:13:07.304ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
492026-07-02T06:13:07.305ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 1.409797ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
502026-07-02T06:13:07.306ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-07-02T06:12:50.467140223Z path = KstatPath { module: "link", instance: 0, name: "tfportrear0_0" } unit = oximeter
512026-07-02T06:13:07.306ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 8977908959309500587 unit = oximeter
522026-07-02T06:13:07.306ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportrear0_0 unit = metrics-task-tracker
532026-07-02T06:13:07.307ZWARNtfportd: tfportqsfp0_0 has no addrs
542026-07-02T06:13:07.609ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 305.182089ms error = "proto error: no records found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
552026-07-02T06:13:08.007ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 689, tv_nsec: 79660045 } }) unit = producer-server
562026-07-02T06:13:08.008ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
572026-07-02T06:13:08.024ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
582026-07-02T06:13:08.270ZINFOtfportd: added new ARP entry entry = 10.151.2.174->52:18:d5:83:ac:18 interface = tfportqsfp0_0
592026-07-02T06:13:08.711ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
602026-07-02T06:13:08.716ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
612026-07-02T06:13:08.720ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
622026-07-02T06:13:08.720ZDEBGtfportd (kstat-sampler): adding new target unit = oximeter
632026-07-02T06:13:08.721ZDEBGtfportd (kstat-sampler-worker): storing new creation time for tracked target creation_time = 2026-07-02T06:12:50.467008761Z path = KstatPath { module: "link", instance: 0, name: "tfportqsfp0_0" } unit = oximeter
642026-07-02T06:13:08.721ZDEBGtfportd (kstat-sampler-worker): inserted empty per-target sample queue id = 13263792732801804906 unit = oximeter
652026-07-02T06:13:08.721ZDEBGtfportd: added new link to kstat sampler link_kind = switch-port-control link_name = tfportqsfp0_0 unit = metrics-task-tracker
662026-07-02T06:13:09.948ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
672026-07-02T06:13:09.953ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
682026-07-02T06:13:09.956ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
692026-07-02T06:13:11.450ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
702026-07-02T06:13:11.455ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
712026-07-02T06:13:11.458ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
722026-07-02T06:13:12.767ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
732026-07-02T06:13:12.772ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
742026-07-02T06:13:12.775ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
752026-07-02T06:13:14.110ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
762026-07-02T06:13:14.115ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
772026-07-02T06:13:14.118ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
782026-07-02T06:13:14.664ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
792026-07-02T06:13:14.668ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
802026-07-02T06:13:14.671ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
812026-07-02T06:13:15.970ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
822026-07-02T06:13:15.976ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
832026-07-02T06:13:15.978ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
842026-07-02T06:13:16.841ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
852026-07-02T06:13:16.846ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
862026-07-02T06:13:16.849ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
872026-07-02T06:13:17.308ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
882026-07-02T06:13:17.472ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
892026-07-02T06:13:17.477ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
902026-07-02T06:13:17.480ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
912026-07-02T06:13:18.724ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
922026-07-02T06:13:18.754ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
932026-07-02T06:13:18.758ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
942026-07-02T06:13:18.762ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
952026-07-02T06:13:20.245ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
962026-07-02T06:13:20.251ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
972026-07-02T06:13:20.254ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
982026-07-02T06:13:21.365ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
992026-07-02T06:13:21.370ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1002026-07-02T06:13:21.373ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1012026-07-02T06:13:22.823ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1022026-07-02T06:13:22.827ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1032026-07-02T06:13:22.830ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1042026-07-02T06:13:22.832ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1052026-07-02T06:13:22.833ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 15.723450983s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1062026-07-02T06:13:23.367ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1072026-07-02T06:13:23.372ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1082026-07-02T06:13:23.375ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1092026-07-02T06:13:23.915ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 705, tv_nsec: 184805808 } }) unit = producer-server
1102026-07-02T06:13:23.916ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
1112026-07-02T06:13:23.924ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1122026-07-02T06:13:24.375ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1132026-07-02T06:13:24.380ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1142026-07-02T06:13:24.382ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1152026-07-02T06:13:25.438ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1162026-07-02T06:13:25.443ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1172026-07-02T06:13:25.446ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1182026-07-02T06:13:25.969ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1192026-07-02T06:13:25.978ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1202026-07-02T06:13:25.981ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1212026-07-02T06:13:27.105ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
1222026-07-02T06:13:27.467ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1232026-07-02T06:13:27.472ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1242026-07-02T06:13:27.475ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1252026-07-02T06:13:28.516ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
1262026-07-02T06:13:28.583ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1272026-07-02T06:13:28.587ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1282026-07-02T06:13:28.590ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1292026-07-02T06:13:29.294ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1302026-07-02T06:13:29.299ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1312026-07-02T06:13:29.301ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1322026-07-02T06:13:29.819ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1332026-07-02T06:13:29.827ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1342026-07-02T06:13:29.831ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1352026-07-02T06:13:31.290ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1362026-07-02T06:13:31.295ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1372026-07-02T06:13:31.298ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1382026-07-02T06:13:32.636ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1392026-07-02T06:13:32.642ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1402026-07-02T06:13:32.651ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1412026-07-02T06:13:33.985ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1422026-07-02T06:13:33.992ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1432026-07-02T06:13:34.043ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1442026-07-02T06:13:35.372ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1452026-07-02T06:13:35.379ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1462026-07-02T06:13:35.383ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1472026-07-02T06:13:36.107ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1482026-07-02T06:13:36.112ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1492026-07-02T06:13:36.116ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1502026-07-02T06:13:37.081ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
1512026-07-02T06:13:37.224ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1522026-07-02T06:13:37.231ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1532026-07-02T06:13:37.235ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1542026-07-02T06:13:38.491ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
1552026-07-02T06:13:38.521ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1562026-07-02T06:13:38.532ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1572026-07-02T06:13:38.535ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1582026-07-02T06:13:38.885ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
1592026-07-02T06:13:38.886ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 3 delay = 31.820332622s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
1602026-07-02T06:13:39.728ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1612026-07-02T06:13:39.734ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1622026-07-02T06:13:39.737ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1632026-07-02T06:13:39.961ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 721, tv_nsec: 272867141 } }) unit = producer-server
1642026-07-02T06:13:39.961ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
1652026-07-02T06:13:39.969ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
1662026-07-02T06:13:41.114ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1672026-07-02T06:13:41.122ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1682026-07-02T06:13:41.126ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1692026-07-02T06:13:42.343ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1702026-07-02T06:13:42.348ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1712026-07-02T06:13:42.353ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1722026-07-02T06:13:43.843ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1732026-07-02T06:13:43.849ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1742026-07-02T06:13:43.873ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1752026-07-02T06:13:44.651ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1762026-07-02T06:13:44.662ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1772026-07-02T06:13:44.667ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1782026-07-02T06:13:45.358ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1792026-07-02T06:13:45.364ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1802026-07-02T06:13:45.368ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1812026-07-02T06:13:45.942ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1822026-07-02T06:13:45.949ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1832026-07-02T06:13:45.953ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1842026-07-02T06:13:46.899ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1852026-07-02T06:13:46.905ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1862026-07-02T06:13:46.909ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1872026-07-02T06:13:47.079ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
1882026-07-02T06:13:48.243ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1892026-07-02T06:13:48.248ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1902026-07-02T06:13:48.251ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1912026-07-02T06:13:48.490ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
1922026-07-02T06:13:49.326ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1932026-07-02T06:13:49.331ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1942026-07-02T06:13:49.335ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1952026-07-02T06:13:50.772ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1962026-07-02T06:13:50.777ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
1972026-07-02T06:13:50.780ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
1982026-07-02T06:13:52.056ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
1992026-07-02T06:13:52.062ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2002026-07-02T06:13:52.065ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2012026-07-02T06:13:53.353ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2022026-07-02T06:13:53.359ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2032026-07-02T06:13:53.362ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2042026-07-02T06:13:54.568ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2052026-07-02T06:13:54.575ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2062026-07-02T06:13:54.578ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2072026-07-02T06:13:54.970ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2082026-07-02T06:13:54.971ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 4 delay = 47.906825215s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2092026-07-02T06:13:55.456ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2102026-07-02T06:13:55.464ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2112026-07-02T06:13:55.469ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2122026-07-02T06:13:56.260ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2132026-07-02T06:13:56.265ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2142026-07-02T06:13:56.268ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2152026-07-02T06:13:57.083ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
2162026-07-02T06:13:57.086ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2172026-07-02T06:13:57.094ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2182026-07-02T06:13:57.099ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2192026-07-02T06:13:58.264ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2202026-07-02T06:13:58.270ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2212026-07-02T06:13:58.275ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2222026-07-02T06:13:58.492ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
2232026-07-02T06:13:59.109ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2242026-07-02T06:13:59.115ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2252026-07-02T06:13:59.119ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2262026-07-02T06:14:00.556ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2272026-07-02T06:14:00.564ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2282026-07-02T06:14:00.570ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2292026-07-02T06:14:00.635ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 741, tv_nsec: 947156954 } }) unit = producer-server
2302026-07-02T06:14:00.636ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
2312026-07-02T06:14:00.661ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
2322026-07-02T06:14:07.998ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
2332026-07-02T06:14:08.021ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2342026-07-02T06:14:08.229ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2352026-07-02T06:14:08.279ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2362026-07-02T06:14:08.495ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
2372026-07-02T06:14:08.955ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2382026-07-02T06:14:08.983ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2392026-07-02T06:14:08.987ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2402026-07-02T06:14:15.715ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
2412026-07-02T06:14:15.834ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2422026-07-02T06:14:15.861ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2432026-07-02T06:14:15.878ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 5 delay = 68.813334073s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
2442026-07-02T06:14:15.950ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2452026-07-02T06:14:18.422ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
2462026-07-02T06:14:18.452ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2472026-07-02T06:14:18.497ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
2482026-07-02T06:14:18.520ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2492026-07-02T06:14:18.619ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2502026-07-02T06:14:19.176ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2512026-07-02T06:14:19.184ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2522026-07-02T06:14:19.189ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2532026-07-02T06:14:20.479ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2542026-07-02T06:14:20.487ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2552026-07-02T06:14:20.491ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2562026-07-02T06:14:21.548ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2572026-07-02T06:14:21.554ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2582026-07-02T06:14:21.557ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2592026-07-02T06:14:22.489ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 763, tv_nsec: 798093076 } }) unit = producer-server
2602026-07-02T06:14:22.492ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
2612026-07-02T06:14:22.533ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
2622026-07-02T06:14:22.542ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2632026-07-02T06:14:22.550ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2642026-07-02T06:14:22.554ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2652026-07-02T06:14:23.864ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2662026-07-02T06:14:23.871ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2672026-07-02T06:14:23.876ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2682026-07-02T06:14:24.895ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2692026-07-02T06:14:24.903ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2702026-07-02T06:14:24.907ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2712026-07-02T06:14:25.862ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2722026-07-02T06:14:25.877ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2732026-07-02T06:14:25.882ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2742026-07-02T06:14:26.878ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2752026-07-02T06:14:26.899ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2762026-07-02T06:14:26.905ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2772026-07-02T06:14:27.887ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2782026-07-02T06:14:27.896ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2792026-07-02T06:14:27.902ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2802026-07-02T06:14:28.417ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
2812026-07-02T06:14:28.499ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
2822026-07-02T06:14:29.398ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2832026-07-02T06:14:29.405ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2842026-07-02T06:14:29.409ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2852026-07-02T06:14:30.680ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2862026-07-02T06:14:30.688ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2872026-07-02T06:14:30.693ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2882026-07-02T06:14:31.638ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2892026-07-02T06:14:31.648ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2902026-07-02T06:14:31.651ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2912026-07-02T06:14:33.092ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2922026-07-02T06:14:33.101ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2932026-07-02T06:14:33.107ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2942026-07-02T06:14:34.326ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2952026-07-02T06:14:34.340ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2962026-07-02T06:14:34.346ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
2972026-07-02T06:14:35.254ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
2982026-07-02T06:14:35.263ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
2992026-07-02T06:14:35.266ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3002026-07-02T06:14:36.192ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3012026-07-02T06:14:36.205ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3022026-07-02T06:14:36.210ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3032026-07-02T06:14:37.297ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3042026-07-02T06:14:37.305ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3052026-07-02T06:14:37.309ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3062026-07-02T06:14:37.536ZDEBGtfportd (producer-registration-task): client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }) unit = producer-server
3072026-07-02T06:14:37.537ZWARNtfportd (producer-registration-task): failed to register with Nexus, will retry count = 6 delay = 90.472378557s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/8e30605/oximeter/producer/src/lib.rs:371 unit = producer-server
3082026-07-02T06:14:38.139ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3092026-07-02T06:14:38.148ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3102026-07-02T06:14:38.151ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3112026-07-02T06:14:38.419ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
3122026-07-02T06:14:38.502ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
3132026-07-02T06:14:39.024ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3142026-07-02T06:14:39.029ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3152026-07-02T06:14:39.032ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3162026-07-02T06:14:39.566ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3172026-07-02T06:14:39.572ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3182026-07-02T06:14:39.576ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3192026-07-02T06:14:41.078ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3202026-07-02T06:14:41.084ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3212026-07-02T06:14:41.088ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3222026-07-02T06:14:41.900ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3232026-07-02T06:14:41.905ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3242026-07-02T06:14:41.909ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3252026-07-02T06:14:42.878ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3262026-07-02T06:14:42.885ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3272026-07-02T06:14:42.888ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3282026-07-02T06:14:43.529ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3292026-07-02T06:14:43.537ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3302026-07-02T06:14:43.541ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3312026-07-02T06:14:45.044ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3322026-07-02T06:14:45.054ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3332026-07-02T06:14:45.060ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3342026-07-02T06:14:45.776ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3352026-07-02T06:14:45.784ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3362026-07-02T06:14:45.788ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3372026-07-02T06:14:46.565ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3382026-07-02T06:14:46.573ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3392026-07-02T06:14:46.577ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3402026-07-02T06:14:47.953ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3412026-07-02T06:14:47.959ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3422026-07-02T06:14:47.963ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3432026-07-02T06:14:48.423ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
3442026-07-02T06:14:48.505ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
3452026-07-02T06:14:49.367ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3462026-07-02T06:14:49.373ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3472026-07-02T06:14:49.377ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3482026-07-02T06:14:50.564ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3492026-07-02T06:14:50.569ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3502026-07-02T06:14:50.572ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3512026-07-02T06:14:51.637ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3522026-07-02T06:14:51.642ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3532026-07-02T06:14:51.646ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3542026-07-02T06:14:53.038ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3552026-07-02T06:14:53.043ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3562026-07-02T06:14:53.046ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3572026-07-02T06:14:54.454ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3582026-07-02T06:14:54.459ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3592026-07-02T06:14:54.462ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3602026-07-02T06:14:55.544ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3612026-07-02T06:14:55.551ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3622026-07-02T06:14:55.554ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3632026-07-02T06:14:56.937ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3642026-07-02T06:14:56.944ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3652026-07-02T06:14:56.948ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3662026-07-02T06:14:57.541ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3672026-07-02T06:14:57.547ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3682026-07-02T06:14:57.550ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3692026-07-02T06:14:58.425ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 8977908959309500587 n_samples = 6 unit = oximeter
3702026-07-02T06:14:58.506ZDEBGtfportd (kstat-sampler-worker): pulled samples from target id = 13263792732801804906 n_samples = 6 unit = oximeter
3712026-07-02T06:14:58.755ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3722026-07-02T06:14:58.761ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3732026-07-02T06:14:58.764ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3742026-07-02T06:14:59.433ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3752026-07-02T06:14:59.440ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3762026-07-02T06:14:59.443ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3772026-07-02T06:14:59.494ZDEBGtfportd (internal-dns-resolver): lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 800, tv_nsec: 805503369 } }) unit = producer-server
3782026-07-02T06:14:59.495ZDEBGtfportd (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221 unit = producer-server
3792026-07-02T06:14:59.507ZDEBGtfportd (producer-registration-task): client request body = Some(Body) method = POST unit = producer-server uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
3802026-07-02T06:14:59.562ZDEBGtfportd (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "cd777f01-c2c3-4168-9e23-c011413dcb30", "content-length": "41", "date": "Thu, 02 Jul 2026 06:14:59 GMT"} }) unit = producer-server
3812026-07-02T06:14:59.564ZDEBGtfportd (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s unit = producer-server wait_period = 150s
3822026-07-02T06:15:00.514ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3832026-07-02T06:15:00.521ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3842026-07-02T06:15:00.526ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3852026-07-02T06:15:01.288ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3862026-07-02T06:15:01.297ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3872026-07-02T06:15:01.302ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]
3882026-07-02T06:15:01.863ZDEBGtfportd: found simports [ "tfportrear0_0", "tfportqsfp0_0", ]
3892026-07-02T06:15:01.872ZDEBGtfportd: sync tfportrear0_0 addrs [fe80::4c44:5cff:fe14:44a3]
3902026-07-02T06:15:01.877ZDEBGtfportd: sync tfportqsfp0_0 addrs [10.151.2.174]