| 1 | | [ May 14 04:08:38 Disabled. ] |
| 2 | | [ May 14 04:08:38 Rereading configuration. ] |
| 3 | | [ May 14 04:08:51 Rereading configuration. ] |
| 4 | | [ May 14 04:08:51 Rereading configuration. ] |
| 5 | | [ May 14 04:08:51 Enabled. ] |
| 6 | | [ May 14 04:08:51 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/mg-ddm/pkg/ddm_method_script.sh &"). ] |
| 7 | | [ May 14 04:08:51 Method "start" exited with status 0. ] |
| 8 | 2024-05-14T04:08:51.457Z | INFO | slog-rs: signal handler waiting for context
|
| 9 | 2024-05-14T04:08:51.474Z | INFO | slog-rs: signal handler waiting got context
|
| 10 | 2024-05-14T04:08:51.475Z | INFO | slog-rs: [tfportrear0_0] sm initialized with addr fe80::4434:52ff:fea2:5e6b on if tfportrear0_0 index 3
|
| 11 | 2024-05-14T04:08:51.475Z | DEBG | slog-rs: [tfportrear0_0] starting discovery handler
|
| 12 | 2024-05-14T04:08:51.476Z | INFO | slog-rs: admin: listening on [::1]:8000
|
| 13 | | [ May 14 04:08:52 Rereading configuration. ] |
| 14 | | [ May 14 04:08:52 Executing refresh method ("/opt/oxide/mg-ddm/pkg/refresh_ctrun_child.sh -c 445"). ] |
| 15 | | + CONTRACT='' |
| 16 | | + getopts c: o |
| 17 | | + CONTRACT=445 |
| 18 | | + getopts c: o |
| 19 | | + [[ -n 445 ]] |
| 20 | | + pgrep -c 445 |
| 21 | | + PARENT_PID=4245 |
| 22 | | + [[ -n 4245 ]] |
| 23 | | + pgrep -P 4245 |
| 24 | | + CHILD_PID=4246 |
| 25 | | + [[ -n 4246 ]] |
| 26 | | + kill -USR1 4246 |
| 27 | 2024-05-14T04:08:52.792Z | INFO | slog-rs: handling smf refresh
|
| 28 | | [ May 14 04:08:52 Method "refresh" exited with status 0. ] |
| 29 | 2024-05-14T04:08:52.794Z | INFO | slog-rs: starting stats server on smf refresh
|
| 30 | 2024-05-14T04:08:52.794Z | INFO | slog-rs: resolving nexus
|
| 31 | 2024-05-14T04:08:52.794Z | INFO | slog-rs: new DNS resolver
addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53]
|
| 32 | 2024-05-14T04:08:52.797Z | DEBG | slog-rs: lookup_socket_v6 srv
dns_name = _nexus._tcp.control-plane.oxide.internal
response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("4db81567-f1e3-4377-871b-92d735132e59.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("d4fac703-f5ba-4b34-aac1-7fc066e8abb3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("ef1c9f55-f835-41e6-8d3a-81e26da276c7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ef1c9f55-f835-41e6-8d3a-81e26da276c7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 6357, tv_nsec: 313268199 } })
|
| 33 | | May 14 04:08:52.799 DEBG registered DTrace probes |
| 34 | | May 14 04:08:52.799 DEBG registered endpoint, path: /{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot |
| 35 | | May 14 04:08:52.799 INFO listening, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:195 |
| 36 | | May 14 04:08:52.799 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:8001, component: dropshot |
| 37 | | May 14 04:08:52.800 DEBG Nexus IP provided explicitly, will use it, addr: [fd00:1122:3344:101::a]:12221 |
| 38 | | May 14 04:08:52.800 DEBG starting producer registration task |
| 39 | | May 14 04:08:52.800 INFO starting oximeter metric producer server, interval: 1s, address: [fd00:1122:3344:101::2]:8001, producer_id: 3af269ad-01d6-459d-9e07-ee70f131397d, route: //3af269ad-01d6-459d-9e07-ee70f131397d, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:284 |
| 40 | | May 14 04:08:52.800 DEBG registering / renewing oximeter producer lease with Nexus, component: producer-registration-task |
| 41 | | May 14 04:08:52.800 DEBG using nexus address for registration, addr: [fd00:1122:3344:101::a]:12221, component: producer-registration-task |
| 42 | | May 14 04:08:52.838 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task |
| 43 | 2024-05-14T04:08:53.476Z | INFO | slog-rs: [tfportrear0_0] nbr is fe80::c84c:a4ff:fe78:a278@buskin server
|
| 44 | 2024-05-14T04:08:53.476Z | DEBG | slog-rs: [tfportrear0_0] transition solicit -> exchange
|
| 45 | 2024-05-14T04:08:53.476Z | INFO | slog-rs: [tfportrear0_0] exchange: listening on [fe80::4434:52ff:fea2:5e6b]:56797
|
| 46 | 2024-05-14T04:08:53.476Z | INFO | slog-rs: [tfportrear0_0] waiting for exchange server to start
|
| 47 | 2024-05-14T04:08:53.478Z | INFO | slog-rs: [tfportrear0_0] sending 5 routes to dendrite
|
| 48 | 2024-05-14T04:08:53.478Z | DEBG | slog-rs: [tfportrear0_0] sending to dpd host=localhost port=12224
|
| 49 | 2024-05-14T04:08:53.510Z | INFO | slog-rs: [tfportrear0_0] adding route fd00:1122:3344:101:: -> fe80::c84c:a4ff:fe78:a278 on port Rear(RearPort(0))/0
|
| 50 | 2024-05-14T04:08:53.511Z | INFO | slog-rs: [tfportrear0_0] adding route fd00:1122:3344:2:: -> fe80::c84c:a4ff:fe78:a278 on port Rear(RearPort(0))/0
|
| 51 | 2024-05-14T04:08:53.511Z | INFO | slog-rs: [tfportrear0_0] adding route fdb0:18c0:4d0d:9fb2:: -> fe80::c84c:a4ff:fe78:a278 on port Rear(RearPort(0))/0
|
| 52 | 2024-05-14T04:08:53.511Z | INFO | slog-rs: [tfportrear0_0] adding route fd00:1122:3344:1:: -> fe80::c84c:a4ff:fe78:a278 on port Rear(RearPort(0))/0
|
| 53 | 2024-05-14T04:08:53.511Z | INFO | slog-rs: [tfportrear0_0] adding route fd00:1122:3344:3:: -> fe80::c84c:a4ff:fe78:a278 on port Rear(RearPort(0))/0
|
| 54 | 2024-05-14T04:08:53.512Z | INFO | slog-rs: [tfportrear0_0] removing routes 0 from dendrite
|
| 55 | 2024-05-14T04:08:53.541Z | DEBG | slog-rs: [tfportrear0_0] redistributing update to 0 peers
|
| 56 | 2024-05-14T04:08:53.867Z | INFO | slog-rs: advertise tunnel: {
TunnelOrigin {
overlay_prefix: V4(
Ipv4Prefix {
addr: 0.0.0.0,
len: 0,
},
),
boundary_addr: fdec:66c:c791:8ffe::1,
vni: 99,
metric: 18446744073709551615,
},
}
|
| 57 | 2024-05-14T04:08:53.868Z | INFO | slog-rs: advertise tunnel: {
TunnelOrigin {
overlay_prefix: V4(
Ipv4Prefix {
addr: 0.0.0.0,
len: 0,
},
),
boundary_addr: fdec:66c:c791:8ffe::1,
vni: 99,
metric: 18446744073709551615,
},
}
|
| 58 | | May 14 04:09:07.838 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task |
| 59 | | May 14 04:09:07.839 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out", delay: 147.025665ms, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426 |
| 60 | | May 14 04:09:08.017 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task |
| 61 | 2024-05-14T04:09:13.705Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 62 | 2024-05-14T04:09:15.706Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 63 | 2024-05-14T04:09:17.707Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 64 | 2024-05-14T04:09:19.708Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 65 | 2024-05-14T04:09:21.709Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 66 | | May 14 04:09:23.219 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task |
| 67 | | May 14 04:09:23.220 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out", delay: 338.159491ms, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426 |
| 68 | | May 14 04:09:23.587 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task |
| 69 | 2024-05-14T04:09:25.711Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 70 | 2024-05-14T04:09:27.712Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 71 | 2024-05-14T04:09:29.713Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 72 | 2024-05-14T04:09:31.714Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 73 | 2024-05-14T04:09:33.715Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 74 | 2024-05-14T04:09:37.717Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 75 | | May 14 04:09:38.548 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task |
| 76 | | May 14 04:09:38.549 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out", delay: 990.685183ms, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426 |
| 77 | | May 14 04:09:39.567 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task |
| 78 | 2024-05-14T04:09:39.718Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 79 | 2024-05-14T04:09:41.719Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 80 | 2024-05-14T04:09:43.720Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 81 | 2024-05-14T04:09:45.721Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 82 | 2024-05-14T04:09:49.723Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 83 | | May 14 04:09:54.528 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task |
| 84 | | May 14 04:09:54.529 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out", delay: 2.913368867s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426 |
| 85 | 2024-05-14T04:09:55.726Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 86 | | May 14 04:09:57.465 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task |
| 87 | 2024-05-14T04:09:57.727Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 88 | 2024-05-14T04:09:59.728Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 89 | 2024-05-14T04:10:01.729Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 90 | 2024-05-14T04:10:03.730Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 91 | 2024-05-14T04:10:05.731Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 92 | 2024-05-14T04:10:07.732Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 93 | 2024-05-14T04:10:09.733Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 94 | 2024-05-14T04:10:11.734Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 95 | | May 14 04:10:12.426 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task |
| 96 | | May 14 04:10:12.427 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out", delay: 5.879585471s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426 |
| 97 | 2024-05-14T04:10:13.735Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 98 | 2024-05-14T04:10:15.736Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 99 | 2024-05-14T04:10:17.737Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 100 | | May 14 04:10:18.321 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task |
| 101 | 2024-05-14T04:10:19.738Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 102 | 2024-05-14T04:10:21.739Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 103 | 2024-05-14T04:10:23.740Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 104 | 2024-05-14T04:10:25.741Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 105 | 2024-05-14T04:10:27.742Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 106 | 2024-05-14T04:10:29.743Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 107 | 2024-05-14T04:10:31.744Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 108 | | May 14 04:10:33.281 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task |
| 109 | | May 14 04:10:33.282 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out", delay: 4.194723857s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426 |
| 110 | 2024-05-14T04:10:33.745Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 111 | 2024-05-14T04:10:35.746Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 112 | | May 14 04:10:37.495 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task |
| 113 | 2024-05-14T04:10:37.747Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 114 | 2024-05-14T04:10:39.748Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 115 | 2024-05-14T04:10:41.749Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 116 | 2024-05-14T04:10:43.750Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 117 | 2024-05-14T04:10:45.751Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 118 | 2024-05-14T04:10:47.752Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 119 | 2024-05-14T04:10:49.753Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 120 | 2024-05-14T04:10:51.754Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 121 | | May 14 04:10:52.455 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task |
| 122 | | May 14 04:10:52.456 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out", delay: 8.658528459s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426 |
| 123 | 2024-05-14T04:10:55.757Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 124 | 2024-05-14T04:10:57.758Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 125 | | May 14 04:11:01.122 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task |
| 126 | 2024-05-14T04:11:01.760Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 127 | 2024-05-14T04:11:03.761Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 128 | 2024-05-14T04:11:05.762Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 129 | 2024-05-14T04:11:07.763Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 130 | 2024-05-14T04:11:09.764Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 131 | 2024-05-14T04:11:11.765Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 132 | 2024-05-14T04:11:13.766Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 133 | 2024-05-14T04:11:15.767Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 134 | | May 14 04:11:16.083 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task |
| 135 | | May 14 04:11:16.084 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out", delay: 42.885123151s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426 |
| 136 | 2024-05-14T04:11:17.768Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 137 | 2024-05-14T04:11:19.769Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 138 | 2024-05-14T04:11:21.770Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 139 | 2024-05-14T04:11:23.771Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 140 | 2024-05-14T04:11:25.772Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 141 | 2024-05-14T04:11:27.773Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 142 | 2024-05-14T04:11:31.775Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 143 | 2024-05-14T04:11:33.775Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 144 | | May 14 04:11:58.955 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task |
| 145 | 2024-05-14T04:12:17.592Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 missed solicit interval
|
| 146 | | May 14 04:12:17.616 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task |
| 147 | | May 14 04:12:17.630 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out", delay: 67.00396294s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426 |
| 148 | 2024-05-14T04:12:47.437Z | WARN | slog-rs: [tfportrear0_0] neighbor buskin@fe80::c84c:a4ff:fe78:a278 expire
|
| 149 | 2024-05-14T04:12:47.453Z | WARN | slog-rs: [tfportrear0_0] expiring peer fe80::c84c:a4ff:fe78:a278 due to discovery event
|
| 150 | 2024-05-14T04:12:47.453Z | INFO | slog-rs: [tfportrear0_0] removing routes 5 from dendrite
|
| 151 | 2024-05-14T04:12:47.940Z | INFO | slog-rs: [tfportrear0_0] nbr is fe80::c84c:a4ff:fe78:a278@buskin server
|
| 152 | 2024-05-14T04:12:48.469Z | ERRO | slog-rs: [tfportrear0_0] failed to remove tunnel routes: {} OPTE driver is not attached
|
| 153 | 2024-05-14T04:12:48.469Z | DEBG | slog-rs: [tfportrear0_0] redistributing expire to 0 peers
|
| 154 | 2024-05-14T04:12:48.471Z | DEBG | slog-rs: [tfportrear0_0] transition solicit -> exchange
|
| 155 | 2024-05-14T04:12:48.480Z | INFO | slog-rs: [tfportrear0_0] exchange: listening on [fe80::4434:52ff:fea2:5e6b]:56797
|
| 156 | 2024-05-14T04:12:48.492Z | INFO | slog-rs: [tfportrear0_0] waiting for exchange server to start
|
| 157 | 2024-05-14T04:12:48.557Z | INFO | slog-rs: [tfportrear0_0] sending 5 routes to dendrite
|
| 158 | 2024-05-14T04:12:48.557Z | DEBG | slog-rs: [tfportrear0_0] sending to dpd host=localhost port=12224
|
| 159 | 2024-05-14T04:12:48.676Z | INFO | slog-rs: [tfportrear0_0] adding route fd00:1122:3344:2:: -> fe80::c84c:a4ff:fe78:a278 on port Rear(RearPort(0))/0
|
| 160 | 2024-05-14T04:12:48.681Z | INFO | slog-rs: [tfportrear0_0] adding route fdb0:18c0:4d0d:9fb2:: -> fe80::c84c:a4ff:fe78:a278 on port Rear(RearPort(0))/0
|
| 161 | 2024-05-14T04:12:48.683Z | INFO | slog-rs: [tfportrear0_0] adding route fd00:1122:3344:101:: -> fe80::c84c:a4ff:fe78:a278 on port Rear(RearPort(0))/0
|
| 162 | 2024-05-14T04:12:48.685Z | INFO | slog-rs: [tfportrear0_0] adding route fd00:1122:3344:3:: -> fe80::c84c:a4ff:fe78:a278 on port Rear(RearPort(0))/0
|
| 163 | 2024-05-14T04:12:48.685Z | INFO | slog-rs: [tfportrear0_0] adding route fd00:1122:3344:1:: -> fe80::c84c:a4ff:fe78:a278 on port Rear(RearPort(0))/0
|
| 164 | 2024-05-14T04:12:48.686Z | INFO | slog-rs: [tfportrear0_0] removing routes 0 from dendrite
|
| 165 | 2024-05-14T04:12:48.732Z | DEBG | slog-rs: [tfportrear0_0] redistributing update to 0 peers
|
| 166 | | May 14 04:13:24.716 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST, component: producer-registration-task |
| 167 | | May 14 04:13:24.733 DEBG client response, result: Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 503, headers: {"content-type": "application/json", "x-request-id": "0d6c045d-13da-4e76-92db-5a97001fe8b8", "content-length": "133", "date": "Tue, 14 May 2024 04:13:24 GMT"} }), component: producer-registration-task |
| 168 | | May 14 04:13:24.747 WARN failed to register as a producer with Nexus, will retry, error: "Error Response: status: 503 Service Unavailable; headers: {\"content-type\": \"application/json\", \"x-request-id\": \"0d6c045d-13da-4e76-92db-5a97001fe8b8\", \"content-length\": \"133\", \"date\": \"Tue, 14 May 2024 04:13:24 GMT\"}; value: Error { error_code: Some(\"ServiceNotAvailable\"), message: \"Service Unavailable\", request_id: \"0d6c045d-13da-4e76-92db-5a97001fe8b8\" }", delay: 168.132375946s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426 |