Max level shown:
1 [ Mar 29 05:52:58 Disabled. ]
2 [ Mar 29 05:52:58 Rereading configuration. ]
3 [ Mar 29 05:53:11 Rereading configuration. ]
4 [ Mar 29 05:53:11 Rereading configuration. ]
5 [ Mar 29 05:53:11 Enabled. ]
6 [ Mar 29 05:53:11 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/mgd/pkg/mgd_method_script.sh &"). ]
7 [ Mar 29 05:53:11 Method "start" exited with status 0. ]
82024-03-29T05:53:11.795ZINFOslog-rs: signal handler waiting for context
92024-03-29T05:53:11.816ZINFOslog-rs: signal handler waiting got context
102024-03-29T05:53:11.816ZINFOslog-rs: bgp routers: {}
112024-03-29T05:53:11.816ZINFOslog-rs: bfd peers: []
122024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bfd/peers unit = api-server
132024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /bfd/peers unit = api-server
142024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /bfd/peers/{addr} unit = api-server
152024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/apply unit = api-server
162024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/graceful_shutdown unit = api-server
172024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/imported4 unit = api-server
182024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/message-history unit = api-server
192024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /bgp/neighbor unit = api-server
202024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/neighbor unit = api-server
212024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /bgp/neighbor unit = api-server
222024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/originate4 unit = api-server
232024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/originate4 unit = api-server
242024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /bgp/router unit = api-server
252024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/router unit = api-server
262024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /bgp/router unit = api-server
272024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /bgp/routers unit = api-server
282024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = POST path = /bgp/withdraw4 unit = api-server
292024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = DELETE path = /static/route4 unit = api-server
302024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = GET path = /static/route4 unit = api-server
312024-03-29T05:53:11.820ZDEBGslog-rs: registered endpoint local_addr = [::]:4676 method = PUT path = /static/route4 unit = api-server
322024-03-29T05:53:11.820ZINFOslog-rs: admin: listening on [::]:4676
332024-03-29T05:53:11.820ZINFOslog-rs: listening local_addr = [::]:4676 unit = api-server
342024-03-29T05:53:11.821ZDEBGslog-rs: successfully registered DTrace USDT probes local_addr = [::]:4676 unit = api-server
352024-03-29T05:53:11.890ZERROslog-rs: get originated tunnel endpoints: Communication Error: error sending request for url (http://localhost:8000/originated_tunnel_endpoints): error trying to connect: tcp connect error: Connection refused (os error 146)
36 [ Mar 29 05:53:13 Rereading configuration. ]
37 [ Mar 29 05:53:13 Executing refresh method ("/opt/oxide/mgd/pkg/refresh_ctrun_child.sh -c 429"). ]
38 + CONTRACT=''
39 + getopts c: o
40 + CONTRACT=429
41 + getopts c: o
42 + [[ -n 429 ]]
43 + pgrep -c 429
44 + PARENT_PID=4187
45 + [[ -n 4187 ]]
46 + pgrep -P 4187
47 + CHILD_PID=4188
48 + [[ -n 4188 ]]
49 + kill -USR1 4188
502024-03-29T05:53:13.244ZINFOslog-rs: handling smf refresh
512024-03-29T05:53:13.246ZINFOslog-rs: starting stats server on smf refresh
52 [ Mar 29 05:53:13 Method "refresh" exited with status 0. ]
532024-03-29T05:53:13.246ZINFOslog-rs: resolving nexus
542024-03-29T05:53:13.246ZINFOslog-rs: new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53]
552024-03-29T05:53:13.444ZINFOslog-rs: accepted connection local_addr = [::]:4676 remote_addr = [fd00:1122:3344:101::1]:59098 unit = api-server
562024-03-29T05:53:13.444ZINFOslog-rs: sending notification for change set ChangeSet { generation: 0, import: ImportChangeSet { added: { Route4ImportKey { prefix: Prefix4 { value: 0.0.0.0, length: 0, }, nexthop: 10.151.1.100, id: 0, priority: 100, }, }, removed: {}, }, origin: OriginChangeSet { added: {}, removed: {}, }, }
572024-03-29T05:53:13.444ZINFOslog-rs: request completed latency_us = 281 local_addr = [::]:4676 method = PUT remote_addr = [fd00:1122:3344:101::1]:59098 req_id = bc84f94e-5204-4a15-ad03-08d905ab5376 response_code = 204 unit = api-server uri = /static/route4
582024-03-29T05:53:13.444ZINFOslog-rs: mg-lower: handling rib change generation 0 -> 0: ChangeSet { generation: 0, import: ImportChangeSet { added: { Route4ImportKey { prefix: Prefix4 { value: 0.0.0.0, length: 0, }, nexthop: 10.151.1.100, id: 0, priority: 100, }, }, removed: {}, }, origin: OriginChangeSet { added: {}, removed: {}, }, }
592024-03-29T05:53:13.445ZERROslog-rs: failed to get port for Route4ImportKey { prefix: Prefix4 { value: 0.0.0.0, length: 0 }, nexthop: 10.151.1.100, id: 0, priority: 100 }: "Unable to get route for Route4ImportKey { prefix: Prefix4 { value: 0.0.0.0, length: 0 }, nexthop: 10.151.1.100, id: 0, priority: 100 }: IoError(Os { code: 3, kind: Uncategorized, message: \"No such process\" })"
602024-03-29T05:53:18.250ZDEBGslog-rs: lookup_socket_v6 srv dns_name = _nexus._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("00110943-032d-4124-9b1a-b12f6e105bb1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("33404a22-f9b6-4bb7-9f67-120c445fb407.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("595220f7-5a03-4309-9899-d806864e6ebc.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("595220f7-5a03-4309-9899-d806864e6ebc.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::c)) }], valid_until: Instant { tv_sec: 732, tv_nsec: 143791512 } })
61 Mar 29 05:53:18.252 DEBG registered DTrace probes
62 Mar 29 05:53:18.252 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
63 Mar 29 05:53:18.252 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
64 Mar 29 05:53:18.252 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
65 Mar 29 05:53:18.252 DEBG registering metric server as a producer
66 Mar 29 05:53:18.285 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
67 Mar 29 05:53:33.286 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
68 Mar 29 05:53:33.287 INFO received request to begin graceful shutdown, {"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 59.111618ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:53:33.287247176Z","hostname":"oxz_switch","pid":4188}
69 local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
70 Mar 29 05:53:33.347 DEBG registered DTrace probes
71 Mar 29 05:53:33.347 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
72 Mar 29 05:53:33.347 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
73 Mar 29 05:53:33.347 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
74 Mar 29 05:53:33.348 DEBG registering metric server as a producer
75 Mar 29 05:53:33.376 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
76 Mar 29 05:53:48.378 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
77 Mar 29 05:53:48.379 INFO received request to begin graceful shutdown, local_addr:{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 103.128528ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:53:48.379250487Z","hostname":"oxz_switch","pid":4188}
78 [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
79 Mar 29 05:53:48.483 DEBG registered DTrace probes
80 Mar 29 05:53:48.483 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
81 Mar 29 05:53:48.483 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
82 Mar 29 05:53:48.483 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
83 Mar 29 05:53:48.483 DEBG registering metric server as a producer
84 Mar 29 05:53:48.512 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
85 Mar 29 05:54:03.514 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
86 Mar 29 05:54:03.515 INFO received request to begin graceful shutdown, local_addr: {"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 201.820773ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:54:03.515457425Z","hostname":"oxz_switch","pid":4188}
87 [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
88 Mar 29 05:54:03.718 DEBG registered DTrace probes
89 Mar 29 05:54:03.718 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
90 Mar 29 05:54:03.719 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
91 Mar 29 05:54:03.719 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
92 Mar 29 05:54:03.719 DEBG registering metric server as a producer
93 Mar 29 05:54:03.748 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
94 Mar 29 05:54:18.748 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
95 Mar 29 05:54:18.750 INFO received request to begin graceful shutdown{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 227.074115ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:54:18.750024924Z","hostname":"oxz_switch","pid":4188}
96 , local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
97 Mar 29 05:54:18.978 DEBG registered DTrace probes
98 Mar 29 05:54:18.978 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
99 Mar 29 05:54:18.978 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
100 Mar 29 05:54:18.978 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
101 Mar 29 05:54:18.978 DEBG registering metric server as a producer
102 Mar 29 05:54:19.007 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
103 Mar 29 05:54:34.008 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
104 Mar 29 05:54:34.009 INFO received request to begin graceful shutdown, local_addr: [fd00:{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 297.672095ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:54:34.009913004Z","hostname":"oxz_switch","pid":4188}
105 1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
106 Mar 29 05:54:34.308 DEBG registered DTrace probes
107 Mar 29 05:54:34.308 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
108 Mar 29 05:54:34.308 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
109 Mar 29 05:54:34.308 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
110 Mar 29 05:54:34.308 DEBG registering metric server as a producer
111 Mar 29 05:54:34.337 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
112 Mar 29 05:54:49.338 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
113 Mar 29 05:54:49.339 INFO{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 181.327809ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:54:49.339934424Z","hostname":"oxz_switch","pid":4188}
114 received request to begin graceful shutdown, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
115 Mar 29 05:54:49.522 DEBG registered DTrace probes
116 Mar 29 05:54:49.522 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
117 Mar 29 05:54:49.522 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
118 Mar 29 05:54:49.522 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
119 Mar 29 05:54:49.522 DEBG registering metric server as a producer
120 Mar 29 05:54:49.551 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
121 Mar 29 05:55:04.552 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
122 Mar 29 05:55:04.553 INFO received request to begin graceful shutdown, local_addr: {"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 397.034297ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:55:04.553174128Z","hostname":"oxz_switch","pid":4188}
123 [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
124 Mar 29 05:55:04.951 DEBG registered DTrace probes
125 Mar 29 05:55:04.951 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
126 Mar 29 05:55:04.951 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
127 Mar 29 05:55:04.952 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
128 Mar 29 05:55:04.952 DEBG registering metric server as a producer
129 Mar 29 05:55:04.980 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
130 Mar 29 05:55:19.981 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
131 Mar 29 05:55:19.982 INFO received request to begin graceful shutdown, local_addr: [fd00{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 464.188323ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:55:19.982542791Z","hostname":"oxz_switch","pid":4188}
132 :1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
133 Mar 29 05:55:20.447 DEBG registered DTrace probes
134 Mar 29 05:55:20.447 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
135 Mar 29 05:55:20.448 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
136 Mar 29 05:55:20.448 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
137 Mar 29 05:55:20.448 DEBG registering metric server as a producer
138 Mar 29 05:55:20.477 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
139 Mar 29 05:55:35.478 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
140 Mar 29 05:55:35.479 INFO received request to begin graceful shutdown, local_addr: [fd00:1122:3344:101::{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 371.651594ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:55:35.479062291Z","hostname":"oxz_switch","pid":4188}
141 2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
142 Mar 29 05:55:35.851 DEBG registered DTrace probes
143 Mar 29 05:55:35.851 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
144 Mar 29 05:55:35.852 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
145 Mar 29 05:55:35.852 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
146 Mar 29 05:55:35.852 DEBG registering metric server as a producer
147 Mar 29 05:55:35.881 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
148 Mar 29 05:55:50.882 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
149 Mar 29 05:55:50.883 INFO {"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 649.616851ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:55:50.88343883Z","hostname":"oxz_switch","pid":4188}
150 received request to begin graceful shutdown, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
151 Mar 29 05:55:51.534 DEBG registered DTrace probes
152 Mar 29 05:55:51.534 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
153 Mar 29 05:55:51.535 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
154 Mar 29 05:55:51.535 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
155 Mar 29 05:55:51.535 DEBG registering metric server as a producer
156 Mar 29 05:55:51.564 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
157 Mar 29 05:56:06.565 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
158 Mar 29 05:56:06.566 INFO received request to begin graceful shutdown, local_addr: [fd00:1122{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 635.910577ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:56:06.566728432Z","hostname":"oxz_switch","pid":4188}
159 :3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
160 Mar 29 05:56:07.203 DEBG registered DTrace probes
161 Mar 29 05:56:07.203 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
162 Mar 29 05:56:07.203 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
163 Mar 29 05:56:07.203 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
164 Mar 29 05:56:07.204 DEBG registering metric server as a producer
165 Mar 29 05:56:07.232 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
166 Mar 29 05:56:22.234 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
167 Mar 29 05:56:22.235 INFO received request to begin graceful shutdown{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 770.701822ms","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:56:22.235247767Z","hostname":"oxz_switch","pid":4188}
168 , local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
169 Mar 29 05:56:23.007 DEBG registered DTrace probes
170 Mar 29 05:56:23.007 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
171 Mar 29 05:56:23.007 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
172 Mar 29 05:56:23.007 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
173 Mar 29 05:56:23.007 DEBG registering metric server as a producer
174 Mar 29 05:56:23.036 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
175 Mar 29 05:56:38.037 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
176 Mar 29 05:56:38.038 INFO received request to begin graceful shutdown, local_addr: {"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 1.321031317s","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:56:38.038488213Z","hostname":"oxz_switch","pid":4188}
177 [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
178 Mar 29 05:56:39.360 DEBG registered DTrace probes
179 Mar 29 05:56:39.360 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
180 Mar 29 05:56:39.360 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
181 Mar 29 05:56:39.361 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
182 Mar 29 05:56:39.361 DEBG registering metric server as a producer
183 Mar 29 05:56:39.389 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
184 Mar 29 05:56:54.390 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
185 Mar 29 05:56:54.391 INFO received request to begin graceful shutdown{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 1.406853294s","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:56:54.391844669Z","hostname":"oxz_switch","pid":4188}
186 , local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
187 Mar 29 05:56:55.799 DEBG registered DTrace probes
188 Mar 29 05:56:55.799 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
189 Mar 29 05:56:55.799 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
190 Mar 29 05:56:55.799 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
191 Mar 29 05:56:55.799 DEBG registering metric server as a producer
192 Mar 29 05:56:55.830 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
193 Mar 29 05:57:10.831 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
194 Mar 29 05:57:10.832 INFO received request to begin graceful shutdown{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 1.504304656s","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:57:10.832234125Z","hostname":"oxz_switch","pid":4188}
195 , local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
196 Mar 29 05:57:12.337 DEBG registered DTrace probes
197 Mar 29 05:57:12.337 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
198 Mar 29 05:57:12.337 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
199 Mar 29 05:57:12.338 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
200 Mar 29 05:57:12.338 DEBG registering metric server as a producer
201 Mar 29 05:57:12.366 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
202 Mar 29 05:57:27.368 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
203 Mar 29 05:57:27.369 INFO received request to begin graceful shutdown, local_addr: {"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 1.50057241s","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:57:27.369995223Z","hostname":"oxz_switch","pid":4188}
204 [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
205 Mar 29 05:57:28.871 DEBG registered DTrace probes
206 Mar 29 05:57:28.872 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
207 Mar 29 05:57:28.872 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
208 Mar 29 05:57:28.872 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
209 Mar 29 05:57:28.872 DEBG registering metric server as a producer
210 Mar 29 05:57:28.902 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
211 Mar 29 05:57:43.903 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
212 Mar 29 05:57:43.905 INFO received request to begin graceful shutdown, local_addr:{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 1.239925978s","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:57:43.905692996Z","hostname":"oxz_switch","pid":4188}
213 [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
214 Mar 29 05:57:45.146 DEBG registered DTrace probes
215 Mar 29 05:57:45.146 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
216 Mar 29 05:57:45.146 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
217 Mar 29 05:57:45.146 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
218 Mar 29 05:57:45.147 DEBG registering metric server as a producer
219 Mar 29 05:57:45.176 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
220 Mar 29 05:58:00.177 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
221 Mar 29 05:58:00.178 INFO received request to begin graceful shutdown, {"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 2.69398106s","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:58:00.178669884Z","hostname":"oxz_switch","pid":4188}
222 local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
223 Mar 29 05:58:02.873 DEBG registered DTrace probes
224 Mar 29 05:58:02.873 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
225 Mar 29 05:58:02.874 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
226 Mar 29 05:58:02.874 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
227 Mar 29 05:58:02.874 DEBG registering metric server as a producer
228 Mar 29 05:58:02.902 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
229 Mar 29 05:58:17.904 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
230 Mar 29 05:58:17.905 INFO received request to begin graceful shutdown, local_addr: [fd00{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 2.936647195s","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:58:17.905385698Z","hostname":"oxz_switch","pid":4188}
231 :1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266
232 Mar 29 05:58:20.843 DEBG registered DTrace probes
233 Mar 29 05:58:20.843 DEBG registered endpoint, path: /collect/{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
234 Mar 29 05:58:20.843 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:195
235 Mar 29 05:58:20.843 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
236 Mar 29 05:58:20.843 DEBG registering metric server as a producer
237 Mar 29 05:58:20.873 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::a]:12221/metrics/producers, method: POST
238 Mar 29 05:58:35.874 DEBG client response, result: Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::a)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, source: TimedOut })
239 Mar 29 05:58:35.875 INFO received request to begin graceful shutdown, local_addr: [{"msg":"stats server not created yet, Error registering as metric producer: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out waiting 4.239895545s","v":0,"name":"slog-rs","level":40,"time":"2024-03-29T05:58:35.875830092Z","hostname":"oxz_switch","pid":4188}
240 fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/2d9b906/dropshot/src/server.rs:266