Max level shown:
1 [ Jul 2 06:12:55 Disabled. ]
2 [ Jul 2 06:12:55 Rereading configuration. ]
3 [ Jul 2 06:12:55 Enabled. ]
4 [ Jul 2 06:12:55 Rereading configuration. ]
5 [ Jul 2 06:12:55 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/mg-ddm/pkg/ddm_method_script.sh &"). ]
6 [ Jul 2 06:12:55 Method "start" exited with status 0. ]
72026-07-02T06:12:55.672ZINFOslog-rs: signal handler waiting for context
82026-07-02T06:12:55.693ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
92026-07-02T06:12:55.693ZERROslog-rs: failed to start stats server: underlay address not found
102026-07-02T06:12:55.693ZINFOslog-rs: signal handler waiting got context
112026-07-02T06:12:55.694ZINFOslog-rs: admin: listening on [::1]:8000
122026-07-02T06:12:55.695ZINFOslog-rs: admin: server started
132026-07-02T06:12:56.693ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
142026-07-02T06:12:57.693ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
152026-07-02T06:12:58.693ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
162026-07-02T06:12:59.694ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
172026-07-02T06:13:00.698ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
182026-07-02T06:13:01.696ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
192026-07-02T06:13:02.696ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
202026-07-02T06:13:03.696ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
212026-07-02T06:13:04.696ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
222026-07-02T06:13:05.696ZWARNslog-rs: [] failed to get IPv6 address for interface tfportrear0_0/ll: not found: tfportrear0_0/ll
232026-07-02T06:13:06.696ZINFOslog-rs: [tfportrear0_0] sm initialized with addr fe80::4c44:5cff:fe14:44a3 on if tfportrear0_0 index 3
242026-07-02T06:13:06.696ZDEBGslog-rs: [tfportrear0_0] starting discovery handler
252026-07-02T06:13:06.697ZINFOslog-rs: [tfportrear0_0] nbr is fe80::5848:fcff:fe62:ec9e@sock server
262026-07-02T06:13:06.697ZDEBGslog-rs: [tfportrear0_0] transition solicit -> exchange
272026-07-02T06:13:06.697ZINFOslog-rs: [tfportrear0_0] exchange: listening on [fe80::4c44:5cff:fe14:44a3]:56797
282026-07-02T06:13:06.697ZINFOslog-rs: [tfportrear0_0] waiting for exchange server to start
29 [ Jul 2 06:13:07 Rereading configuration. ]
30 [ Jul 2 06:13:07 Executing refresh method ("/opt/oxide/mg-ddm/pkg/refresh_ctrun_child.sh -c 213"). ]
31 + CONTRACT=''
32 + getopts c: o
33 + CONTRACT=213
34 + getopts c: o
35 + [[ -n 213 ]]
36 + pgrep -c 213
37 + PARENT_PID=2289
38 + [[ -n 2289 ]]
39 + pgrep -P 2289
40 + CHILD_PID=2290
41 + [[ -n 2290 ]]
42 + kill -USR1 2290
432026-07-02T06:13:07.141ZINFOslog-rs: handling smf refresh
44 [ Jul 2 06:13:07 Method "refresh" exited with status 0. ]
452026-07-02T06:13:07.144ZINFOslog-rs: starting stats server on smf refresh
46 Jul 02 06:13:07.145 DEBG registered DTrace probes
47 Jul 02 06:13:07.146 DEBG registered endpoint, versions: all, path: /{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot
48 Jul 02 06:13:07.146 INFO listening, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:335
49 Jul 02 06:13:07.146 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot
50 Jul 02 06:13:07.146 DEBG Nexus IP not provided, will use DNS to resolve it
51 Jul 02 06:13:07.146 INFO new DNS resolver, addresses: [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53], component: internal-dns-resolver, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/internal-dns/resolver/src/resolver.rs:144
52 Jul 02 06:13:07.146 DEBG starting producer registration task
53 Jul 02 06:13:07.147 INFO starting oximeter metric producer server, interval: 1s, address: [fd00:1122:3344:101::2]:8001, producer_id: e16cd9c3-a740-480a-8062-e0979d5d86f5, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:280
54 Jul 02 06:13:07.147 DEBG registering as oximeter producer with Nexus, registration_count: 0, component: producer-registration-task
55 Jul 02 06:13:07.148 WARN failed to register with Nexus, will retry, error: proto error: no records found for Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, delay: 1.542869ms, count: 0, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
56 Jul 02 06:13:07.487 WARN failed to register with Nexus, will retry, error: proto error: no records found for Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, delay: 340.378226ms, count: 1, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
57 Jul 02 06:13:07.887 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 688, tv_nsec: 959153220 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
58 Jul 02 06:13:07.887 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
59 Jul 02 06:13:07.889 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
602026-07-02T06:13:08.327ZINFOslog-rs: advertise tunnel: { TunnelOrigin { overlay_prefix: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), boundary_addr: fdac:73d7:6d1f:5dfa::1, vni: 99, metric: 18446744073709551615, }, }
612026-07-02T06:13:08.699ZWARNslog-rs: [tfportrear0_0] exchange pull: hyper client error: client error (Connect)
622026-07-02T06:13:08.700ZINFOslog-rs: [tfportrear0_0] sending 5 routes to dendrite
632026-07-02T06:13:08.700ZDEBGslog-rs: [tfportrear0_0] sending to dpd host=localhost port=12224
642026-07-02T06:13:08.701ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:1:: -> fe80::5848:fcff:fe62:ec9e on port Rear(Rear("rear0"))/0
652026-07-02T06:13:08.703ZINFOslog-rs: [tfportrear0_0] adding route fdb0:18c0:4d0d:a02a:: -> fe80::5848:fcff:fe62:ec9e on port Rear(Rear("rear0"))/0
662026-07-02T06:13:08.703ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:3:: -> fe80::5848:fcff:fe62:ec9e on port Rear(Rear("rear0"))/0
672026-07-02T06:13:08.703ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:2:: -> fe80::5848:fcff:fe62:ec9e on port Rear(Rear("rear0"))/0
682026-07-02T06:13:08.703ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:101:: -> fe80::5848:fcff:fe62:ec9e on port Rear(Rear("rear0"))/0
692026-07-02T06:13:08.704ZINFOslog-rs: [tfportrear0_0] removing routes 0 from dendrite
702026-07-02T06:13:08.705ZDEBGslog-rs: [tfportrear0_0] redistributing update to 0 peers
71 Jul 02 06:13:22.697 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
72 Jul 02 06:13:22.697 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out, delay: 15.743978463s, count: 2, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
73 Jul 02 06:13:24.097 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 705, tv_nsec: 367067525 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
74 Jul 02 06:13:24.098 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
75 Jul 02 06:13:24.098 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
762026-07-02T06:13:24.504ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::5848:fcff:fe62:ec9e missed solicit interval
772026-07-02T06:13:26.505ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::5848:fcff:fe62:ec9e missed solicit interval
782026-07-02T06:13:28.505ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::5848:fcff:fe62:ec9e missed solicit interval
792026-07-02T06:13:30.505ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::5848:fcff:fe62:ec9e missed solicit interval
802026-07-02T06:13:32.505ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::5848:fcff:fe62:ec9e missed solicit interval
812026-07-02T06:13:34.518ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::5848:fcff:fe62:ec9e missed solicit interval
82 Jul 02 06:13:39.059 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
83 Jul 02 06:13:39.059 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out, delay: 32.150650617s, count: 3, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
84 Jul 02 06:13:40.946 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 722, tv_nsec: 257910785 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
85 Jul 02 06:13:40.946 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
86 Jul 02 06:13:40.947 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
872026-07-02T06:13:44.559ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::5848:fcff:fe62:ec9e missed solicit interval
88 Jul 02 06:13:55.949 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
89 Jul 02 06:13:55.949 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out, delay: 49.041736511s, count: 4, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
90 Jul 02 06:14:00.652 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 741, tv_nsec: 963794639 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
91 Jul 02 06:14:00.653 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
92 Jul 02 06:14:00.655 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
932026-07-02T06:14:05.051ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::5848:fcff:fe62:ec9e missed solicit interval
942026-07-02T06:14:15.124ZWARNslog-rs: [tfportrear0_0] neighbor sock@fe80::5848:fcff:fe62:ec9e expire
952026-07-02T06:14:15.150ZWARNslog-rs: [tfportrear0_0] expiring peer fe80::5848:fcff:fe62:ec9e due to discovery event
962026-07-02T06:14:15.157ZINFOslog-rs: [tfportrear0_0] removing routes 5 from dendrite
972026-07-02T06:14:15.177ZINFOslog-rs: [tfportrear0_0] nbr is fe80::5848:fcff:fe62:ec9e@sock server
98 Jul 02 06:14:15.681 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
99 Jul 02 06:14:15.760 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out, delay: 68.773876384s, count: 5, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
1002026-07-02T06:14:15.855ZERROslog-rs: [tfportrear0_0] failed to remove tunnel routes: {} OPTE driver is not attached
1012026-07-02T06:14:15.855ZDEBGslog-rs: [tfportrear0_0] redistributing expire to 0 peers
1022026-07-02T06:14:15.885ZDEBGslog-rs: [tfportrear0_0] transition solicit -> exchange
1032026-07-02T06:14:15.906ZINFOslog-rs: [tfportrear0_0] exchange: listening on [fe80::4c44:5cff:fe14:44a3]:56797
1042026-07-02T06:14:15.947ZINFOslog-rs: [tfportrear0_0] waiting for exchange server to start
1052026-07-02T06:14:16.056ZINFOslog-rs: [tfportrear0_0] sending 5 routes to dendrite
1062026-07-02T06:14:16.056ZDEBGslog-rs: [tfportrear0_0] sending to dpd host=localhost port=12224
1072026-07-02T06:14:16.064ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:3:: -> fe80::5848:fcff:fe62:ec9e on port Rear(Rear("rear0"))/0
1082026-07-02T06:14:17.845ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:2:: -> fe80::5848:fcff:fe62:ec9e on port Rear(Rear("rear0"))/0
1092026-07-02T06:14:17.850ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:101:: -> fe80::5848:fcff:fe62:ec9e on port Rear(Rear("rear0"))/0
1102026-07-02T06:14:17.853ZINFOslog-rs: [tfportrear0_0] adding route fd00:1122:3344:1:: -> fe80::5848:fcff:fe62:ec9e on port Rear(Rear("rear0"))/0
1112026-07-02T06:14:17.853ZINFOslog-rs: [tfportrear0_0] adding route fdb0:18c0:4d0d:a02a:: -> fe80::5848:fcff:fe62:ec9e on port Rear(Rear("rear0"))/0
1122026-07-02T06:14:17.855ZINFOslog-rs: [tfportrear0_0] removing routes 0 from dendrite
1132026-07-02T06:14:17.856ZDEBGslog-rs: [tfportrear0_0] redistributing update to 0 peers
114 Jul 02 06:14:20.645 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 761, tv_nsec: 955838303 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
115 Jul 02 06:14:20.647 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
116 Jul 02 06:14:20.650 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
117 Jul 02 06:14:35.653 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
118 Jul 02 06:14:35.653 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out, delay: 88.7449397s, count: 6, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
119 Jul 02 06:14:56.222 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("1932ef4c-8532-4db6-9e2a-a9c76a32e899.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("6a7f7c01-d53d-4c32-b764-70db9a6b0544.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e354872d-d25b-4ea4-b14d-12c9be4ccd30.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 797, tv_nsec: 533287470 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
120 Jul 02 06:14:56.223 DEBG will register with Nexus at [fd00:1122:3344:101::a]:12221, component: producer-registration-task
121 Jul 02 06:14:56.224 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task
122 Jul 02 06:14:56.290 DEBG client response, result: Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "16ba7cc9-93a7-47f7-929e-f93f8beb3722", "content-length": "41", "date": "Thu, 02 Jul 2026 06:14:56 GMT"} }), component: producer-registration-task
123 Jul 02 06:14:56.291 DEBG successfully registered with Nexus, pausing until time to renew lease, wait_period: 150s, lease_duration: 600s, component: producer-registration-task
124 Jul 02 06:14:56.342 INFO accepted connection, remote_addr: [fd00:1122:3344:101::d]:51927, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:1057
125 Jul 02 06:14:56.355 INFO request completed, latency_us: 12175, response_code: 200, uri: /e16cd9c3-a740-480a-8062-e0979d5d86f5, method: GET, req_id: 6e368f55-949a-421a-9c96-377de64bf136, remote_addr: [fd00:1122:3344:101::d]:51927, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
126 Jul 02 06:14:57.336 INFO request completed, latency_us: 602, response_code: 200, uri: /e16cd9c3-a740-480a-8062-e0979d5d86f5, method: GET, req_id: 2018e019-0c97-4dd5-9fc8-26204e151624, remote_addr: [fd00:1122:3344:101::d]:51927, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
127 Jul 02 06:14:58.336 INFO request completed, latency_us: 601, response_code: 200, uri: /e16cd9c3-a740-480a-8062-e0979d5d86f5, method: GET, req_id: b0e49cc4-4950-46ed-93ac-990c85c89d91, remote_addr: [fd00:1122:3344:101::d]:51927, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
128 Jul 02 06:14:59.336 INFO request completed, latency_us: 637, response_code: 200, uri: /e16cd9c3-a740-480a-8062-e0979d5d86f5, method: GET, req_id: f30c8fc5-9509-46ca-8afb-7fb7e5d9a56c, remote_addr: [fd00:1122:3344:101::d]:51927, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
129 Jul 02 06:15:00.336 INFO request completed, latency_us: 613, response_code: 200, uri: /e16cd9c3-a740-480a-8062-e0979d5d86f5, method: GET, req_id: d0c29d9b-788b-4651-b4ab-07e71c664a4e, remote_addr: [fd00:1122:3344:101::d]:51927, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
130 Jul 02 06:15:01.336 INFO request completed, latency_us: 721, response_code: 200, uri: /e16cd9c3-a740-480a-8062-e0979d5d86f5, method: GET, req_id: f798f1be-7bb5-4941-b2b8-4df3abf11bb5, remote_addr: [fd00:1122:3344:101::d]:51927, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874