Max level shown:
1 [ May 20 20:39:22 Disabled. ]
2 [ May 20 20:39:22 Rereading configuration. ]
3 [ May 20 20:39:35 Rereading configuration. ]
4 [ May 20 20:39:35 Rereading configuration. ]
5 [ May 20 20:39:35 Enabled. ]
6 [ May 20 20:39:35 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/mgd/pkg/mgd_method_script.sh &"). ]
7 [ May 20 20:39:35 Method "start" exited with status 0. ]
82024-05-20T20:39:35.182ZINFOslog-rs: signal handler waiting for context
92024-05-20T20:39:35.205ZINFOslog-rs: signal handler waiting got context
102024-05-20T20:39:35.205ZINFOslog-rs: bgp routers: {}
112024-05-20T20:39:35.205ZINFOslog-rs: bfd peers: []
122024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bfd/peers unit = api-server
132024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /bfd/peers unit = api-server
142024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /bfd/peers/{addr} unit = api-server
152024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /bgp/config/checker unit = api-server
162024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/config/checker unit = api-server
172024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/config/checker unit = api-server
182024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /bgp/config/checker unit = api-server
192024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /bgp/config/neighbor unit = api-server
202024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/config/neighbor unit = api-server
212024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/config/neighbor unit = api-server
222024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /bgp/config/neighbor unit = api-server
232024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/config/neighbors unit = api-server
242024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /bgp/config/origin4 unit = api-server
252024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/config/origin4 unit = api-server
262024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/config/origin4 unit = api-server
272024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /bgp/config/origin4 unit = api-server
282024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /bgp/config/router unit = api-server
292024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/config/router unit = api-server
302024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/config/router unit = api-server
312024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /bgp/config/router unit = api-server
322024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/config/routers unit = api-server
332024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /bgp/config/shaper unit = api-server
342024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/config/shaper unit = api-server
352024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/config/shaper unit = api-server
362024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /bgp/config/shaper unit = api-server
372024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/message-history unit = api-server
382024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/omicron/apply unit = api-server
392024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/status/imported unit = api-server
402024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/status/neighbors unit = api-server
412024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/status/selected unit = api-server
422024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /static/route4 unit = api-server
432024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /static/route4 unit = api-server
442024-05-20T20:39:35.209ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /static/route4 unit = api-server
452024-05-20T20:39:35.209ZINFOslog-rs: admin: listening on [::]:4676
462024-05-20T20:39:35.209ZINFOslog-rs: listening local_addr = [::]:4676 unit = api-server
472024-05-20T20:39:35.210ZDEBGslog-rs: successfully registered DTrace USDT probes local_addr = [::]:4676 unit = api-server
482024-05-20T20:39:35.280ZERROslog-rs: get originated tunnel endpoints: Communication Error: error sending request for url (http://localhost:8000/originated_tunnel_endpoints): error trying to connect: tcp connect error: Connection refused (os error 146)
49 [ May 20 20:39:36 Rereading configuration. ]
50 [ May 20 20:39:36 Executing refresh method ("/opt/oxide/mgd/pkg/refresh_ctrun_child.sh -c 435"). ]
51 + CONTRACT=''
52 + getopts c: o
53 + CONTRACT=435
54 + getopts c: o
55 + [[ -n 435 ]]
56 + pgrep -c 435
57 + PARENT_PID=4226
58 + [[ -n 4226 ]]
59 + pgrep -P 4226
60 + CHILD_PID=4227
61 + [[ -n 4227 ]]
62 + kill -USR1 4227
632024-05-20T20:39:36.645ZINFOslog-rs: handling smf refresh
642024-05-20T20:39:36.646ZINFOslog-rs: starting stats server on smf refresh
65 [ May 20 20:39:36 Method "refresh" exited with status 0. ]
662024-05-20T20:39:36.647ZINFOslog-rs: resolving nexus
672024-05-20T20:39:36.647ZINFOslog-rs: new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53]
682024-05-20T20:39:36.900ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::1]:36948 unit = api-server
692024-05-20T20:39:36.901ZINFOslog-rs: request completed latency_us = 355 local_addr = [::]:4676 method = PUT remote_addr = [fd00:1122:3344:101::1]:36948 req_id = 11d6c48a-fccd-4f9e-94d7-1dafc3b45298 response_code = 204 unit = api-server uri = /static/route4
702024-05-20T20:39:36.901ZERROslog-rs: handling change failed: libnet error route io error No such process (os error 3)
712024-05-20T20:39:36.901ZINFOslog-rs: restarting sync loop
722024-05-20T20:39:41.649ZWARNslog-rs: resolving nexus failed, no record found for Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } waiting 53.585641ms
732024-05-20T20:39:41.704ZDEBGslog-rs: lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("14030859-ed0f-47ae-9477-941c8fd9b402.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2f0b1227-85e6-4def-9a56-3028cc1a9e6b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("baf5303b-2c5b-4e5f-aa55-2481f975633b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("baf5303b-2c5b-4e5f-aa55-2481f975633b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 749, tv_nsec: 944898740 } })
74 May 20 20:39:41.706 DEBG registered DTrace probes
75 May 20 20:39:41.706 DEBG registered endpoint, path: /{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
76 May 20 20:39:41.706 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:195
77 May 20 20:39:41.706 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
78 May 20 20:39:41.706 DEBG Nexus IP provided explicitly, will use it, addr: [fd00:1122:3344:101::b]:12221
79 May 20 20:39:41.706 DEBG starting producer registration task
80 May 20 20:39:41.706 INFO starting oximeter metric producer server, interval: 1s, address: [fd00:1122:3344:101::2]:4677, producer_id: c03b05bf-8f39-4e65-af0d-0a482255c81e, route: //c03b05bf-8f39-4e65-af0d-0a482255c81e, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:284
81 May 20 20:39:41.707 DEBG registering / renewing oximeter producer lease with Nexus, component: producer-registration-task
82 May 20 20:39:41.707 DEBG using nexus address for registration, addr: [fd00:1122:3344:101::b]:12221, component: producer-registration-task
83 May 20 20:39:41.738 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::b]:12221/metrics/producers, method: POST, component: producer-registration-task
84 May 20 20:39:57.209 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task
85 May 20 20:39:57.211 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out", delay: 135.076933ms, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426
86 May 20 20:39:57.377 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::b]:12221/metrics/producers, method: POST, component: producer-registration-task
87 May 20 20:40:12.412 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task
88 May 20 20:40:12.414 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out", delay: 257.263473ms, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426
89 May 20 20:40:12.703 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::b]:12221/metrics/producers, method: POST, component: producer-registration-task
90 May 20 20:40:27.704 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task
91 May 20 20:40:27.705 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out", delay: 1.257260919s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426
92 May 20 20:40:28.994 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::b]:12221/metrics/producers, method: POST, component: producer-registration-task
93 May 20 20:40:43.995 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task
94 May 20 20:40:43.997 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out", delay: 1.702638797s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426
95 May 20 20:40:45.732 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::b]:12221/metrics/producers, method: POST, component: producer-registration-task
96 May 20 20:41:00.733 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task
97 May 20 20:41:00.734 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out", delay: 2.034057051s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426
98 May 20 20:41:02.800 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::b]:12221/metrics/producers, method: POST, component: producer-registration-task
99 May 20 20:41:17.801 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task
100 May 20 20:41:17.803 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out", delay: 11.9037831s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426
101 May 20 20:41:29.737 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::b]:12221/metrics/producers, method: POST, component: producer-registration-task
102 May 20 20:41:44.739 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task
103 May 20 20:41:44.740 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out", delay: 17.423788144s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426
104 May 20 20:42:02.196 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::b]:12221/metrics/producers, method: POST, component: producer-registration-task
105 May 20 20:42:17.200 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task
106 May 20 20:42:17.202 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out", delay: 45.719926193s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426
107 May 20 20:43:03.350 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::b]:12221/metrics/producers, method: POST, component: producer-registration-task
108 May 20 20:43:18.352 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut }), component: producer-registration-task
109 May 20 20:43:18.354 WARN failed to register as a producer with Nexus, will retry, error: "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out", delay: 86.777303383s, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-d039c41f152bda83/ee901b0/oximeter/producer/src/lib.rs:426
1102024-05-20T20:43:28.182ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::c]:50042 unit = api-server
1112024-05-20T20:43:28.182ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::b]:61351 unit = api-server
1122024-05-20T20:43:28.187ZINFOslog-rs: request completed latency_us = 3759 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::b]:61351 req_id = 2aa37de6-edc3-404c-a3f2-abe281ab4ff1 response_code = 200 unit = api-server uri = /bfd/peers
1132024-05-20T20:43:28.187ZINFOslog-rs: request completed latency_us = 3768 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::c]:50042 req_id = a066961a-c0a7-476f-afa5-c8a016b930bc response_code = 200 unit = api-server uri = /bfd/peers
1142024-05-20T20:43:28.201ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::a]:55448 unit = api-server
1152024-05-20T20:43:28.202ZINFOslog-rs: request completed latency_us = 36 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::a]:55448 req_id = c3823fc1-65fc-4058-b035-18e2a1d7d0be response_code = 200 unit = api-server uri = /bfd/peers
1162024-05-20T20:43:28.947ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::b]:43506 unit = api-server
1172024-05-20T20:43:28.947ZINFOslog-rs: request completed latency_us = 30 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::b]:43506 req_id = 69ba1449-057f-4e1a-921b-09a12ec09d09 response_code = 200 unit = api-server uri = /bfd/peers
1182024-05-20T20:43:58.165ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::c]:37054 unit = api-server
1192024-05-20T20:43:58.165ZINFOslog-rs: request completed latency_us = 34 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::c]:37054 req_id = d6456bc2-ccbc-4700-9c5b-7e69072445a0 response_code = 200 unit = api-server uri = /bfd/peers
1202024-05-20T20:43:58.167ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::a]:41027 unit = api-server
1212024-05-20T20:43:58.168ZINFOslog-rs: request completed latency_us = 220 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::a]:41027 req_id = d195328c-a267-43ea-b235-79e91759a1c8 response_code = 200 unit = api-server uri = /bfd/peers
1222024-05-20T20:43:58.270ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::b]:36601 unit = api-server
1232024-05-20T20:43:58.270ZINFOslog-rs: request completed latency_us = 21 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::b]:36601 req_id = 756fc64a-3a72-43e9-a459-99267300d5b0 response_code = 200 unit = api-server uri = /bfd/peers
1242024-05-20T20:43:58.305ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::a]:43371 unit = api-server
1252024-05-20T20:43:58.305ZINFOslog-rs: request completed error_message_external = Internal Server Error error_message_internal = key must be a string latency_us = 367 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::a]:43371 req_id = 78a4e777-7f8a-4bf4-82c5-986b89161f20 response_code = 500 unit = api-server uri = /static/route4
1262024-05-20T20:43:58.318ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::b]:45678 unit = api-server
1272024-05-20T20:43:58.318ZINFOslog-rs: request completed error_message_external = Internal Server Error error_message_internal = key must be a string latency_us = 17 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::b]:45678 req_id = 2813ac18-658a-4383-88e7-c203ff9494ca response_code = 500 unit = api-server uri = /static/route4
1282024-05-20T20:43:58.337ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::c]:55321 unit = api-server
1292024-05-20T20:43:58.337ZINFOslog-rs: request completed error_message_external = Internal Server Error error_message_internal = key must be a string latency_us = 15 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::c]:55321 req_id = 7d5b1bcc-b073-4666-af79-0302479815a4 response_code = 500 unit = api-server uri = /static/route4
1302024-05-20T20:44:28.253ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::c]:58871 unit = api-server
1312024-05-20T20:44:28.253ZINFOslog-rs: request completed latency_us = 32 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::c]:58871 req_id = b50fb64f-38fd-4977-a504-a435eafebc8a response_code = 200 unit = api-server uri = /bfd/peers
1322024-05-20T20:44:28.379ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::a]:35228 unit = api-server
1332024-05-20T20:44:28.379ZINFOslog-rs: request completed latency_us = 25 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::a]:35228 req_id = 80d43a0d-7dbf-4fbc-8b1e-6de174838dfb response_code = 200 unit = api-server uri = /bfd/peers
1342024-05-20T20:44:28.437ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::b]:38418 unit = api-server
1352024-05-20T20:44:28.437ZINFOslog-rs: request completed latency_us = 838 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::b]:38418 req_id = bad0d99d-ee42-4868-9901-3b93387cdc68 response_code = 200 unit = api-server uri = /bfd/peers
1362024-05-20T20:44:28.519ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::a]:56443 unit = api-server
1372024-05-20T20:44:28.520ZINFOslog-rs: request completed error_message_external = Internal Server Error error_message_internal = key must be a string latency_us = 670 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::a]:56443 req_id = 2b4dc189-e10e-4d96-b6b1-c7c9f1b6b4eb response_code = 500 unit = api-server uri = /static/route4
1382024-05-20T20:44:28.531ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::b]:61367 unit = api-server
1392024-05-20T20:44:28.531ZINFOslog-rs: request completed error_message_external = Internal Server Error error_message_internal = key must be a string latency_us = 643 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::b]:61367 req_id = d9e28b19-ac18-4461-be98-3d2f3a0fbbb0 response_code = 500 unit = api-server uri = /static/route4
1402024-05-20T20:44:28.634ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::c]:63433 unit = api-server
1412024-05-20T20:44:28.634ZINFOslog-rs: request completed error_message_external = Internal Server Error error_message_internal = key must be a string latency_us = 23 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::c]:63433 req_id = a55db0ce-a916-4bbe-8dc7-839f1a2f902e response_code = 500 unit = api-server uri = /static/route4
142 May 20 20:44:45.159 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::b]:12221/metrics/producers, method: POST, component: producer-registration-task
143 May 20 20:44:45.226 DEBG client response, result: Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "d2fe8837-22c4-4075-bf0c-b0876be04df2", "content-length": "41", "date": "Mon, 20 May 2024 20:44:45 GMT"} }), component: producer-registration-task
144 May 20 20:44:45.227 DEBG registered with nexus successfully, component: producer-registration-task
145 May 20 20:44:45.227 DEBG pausing until time to renew lease, wait_period: 150s, lease_duration: 600s, component: producer-registration-task
146 May 20 20:44:46.257 INFO accepted connection, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:765
147 May 20 20:44:46.264 INFO request completed, latency_us: 6657, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 1ee7a2b9-ee1d-4f1c-8fa8-b005b9043829, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
148 May 20 20:44:47.256 INFO request completed, latency_us: 101, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 286acc65-7bed-4066-9532-3b829eb03a78, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
149 May 20 20:44:48.256 INFO request completed, latency_us: 97, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 1c5792b8-0d94-4f8a-9a27-6628c05c9199, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
150 May 20 20:44:49.255 INFO request completed, latency_us: 96, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 08811413-be9a-4cab-ba4d-429ed724fa95, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
151 May 20 20:44:50.256 INFO request completed, latency_us: 86, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: fe5505f2-df90-44cc-a632-2482e6b0836b, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
152 May 20 20:44:51.256 INFO request completed, latency_us: 90, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 60bd4f30-958b-4bc4-a72d-edd8f19f9748, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
153 May 20 20:44:52.256 INFO request completed, latency_us: 92, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 365a81a5-ddc4-4c88-ab24-475df1d21795, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
154 May 20 20:44:53.256 INFO request completed, latency_us: 92, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 399a7be7-9e8a-4eec-b17b-f568f8ae2f20, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
155 May 20 20:44:54.256 INFO request completed, latency_us: 954, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 44433019-6bec-421e-86e8-eace2e1c64f0, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
156 May 20 20:44:55.256 INFO request completed, latency_us: 113, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 4de4b13f-4a22-4a6c-8219-0b1ee6490822, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
157 May 20 20:44:56.256 INFO request completed, latency_us: 90, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 81310e4a-f297-4803-b304-432790440cf2, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
158 May 20 20:44:57.256 INFO request completed, latency_us: 89, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 86c2c07f-ed8b-4aee-a36f-0b96a5f36e08, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
1592024-05-20T20:44:58.137ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::c]:63471 unit = api-server
1602024-05-20T20:44:58.138ZINFOslog-rs: request completed latency_us = 676 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::c]:63471 req_id = 3ab4fbf3-5bca-40ca-851f-bd6013815bcb response_code = 200 unit = api-server uri = /bfd/peers
1612024-05-20T20:44:58.178ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::a]:55644 unit = api-server
1622024-05-20T20:44:58.178ZINFOslog-rs: request completed latency_us = 21 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::a]:55644 req_id = 8b3cb653-d33e-42b0-b502-816f38f3ff34 response_code = 200 unit = api-server uri = /bfd/peers
1632024-05-20T20:44:58.189ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::b]:34614 unit = api-server
1642024-05-20T20:44:58.189ZINFOslog-rs: request completed latency_us = 11 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::b]:34614 req_id = 9c3c0835-1ab4-4204-a620-c95ba0d22638 response_code = 200 unit = api-server uri = /bfd/peers
1652024-05-20T20:44:58.254ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::a]:33606 unit = api-server
1662024-05-20T20:44:58.254ZINFOslog-rs: request completed error_message_external = Internal Server Error error_message_internal = key must be a string latency_us = 24 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::a]:33606 req_id = dcb98896-d648-4d74-a6f5-af7f3dfc86a0 response_code = 500 unit = api-server uri = /static/route4
167 May 20 20:44:58.256 INFO request completed, latency_us: 74, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 5639c0da-6caa-46ec-ab30-54245f8fa19a, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
1682024-05-20T20:44:58.261ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::b]:65452 unit = api-server
1692024-05-20T20:44:58.261ZINFOslog-rs: request completed error_message_external = Internal Server Error error_message_internal = key must be a string latency_us = 13 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::b]:65452 req_id = 08f7dbcf-4b18-4b34-9b50-41879ab371b1 response_code = 500 unit = api-server uri = /static/route4
1702024-05-20T20:44:58.269ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::c]:35931 unit = api-server
1712024-05-20T20:44:58.269ZINFOslog-rs: request completed error_message_external = Internal Server Error error_message_internal = key must be a string latency_us = 15 local_addr = [::]:4676 method = GET remote_addr = [fd00:1122:3344:101::c]:35931 req_id = c0d7c3c8-a6c0-4279-9f3d-817d8478d96b response_code = 500 unit = api-server uri = /static/route4
172 May 20 20:44:59.255 INFO request completed, latency_us: 99, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 3df8ae3a-69a3-496c-bedb-559409a53ba7, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849
173 May 20 20:45:00.256 INFO request completed, latency_us: 90, response_code: 200, uri: /c03b05bf-8f39-4e65-af0d-0a482255c81e, method: GET, req_id: 364ca41a-2fb3-439f-802c-5cf7ecf73a70, remote_addr: [fd00:1122:3344:101::d]:54423, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/33e158b/dropshot/src/server.rs:849