Max level shown:
1 [ Oct 27 05:26:09 Disabled. ]
2 [ Oct 27 05:26:09 Rereading configuration. ]
3 [ Oct 27 05:26:09 Enabled. ]
4 [ Oct 27 05:26:09 Rereading configuration. ]
5 [ Oct 27 05:26:09 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/mgd/pkg/mgd_method_script.sh &"). ]
6 [ Oct 27 05:26:09 Method "start" exited with status 0. ]
72026-10-27T05:26:09.928ZINFOslog-rs (mgd): signal handler waiting for context module = admin unit = signal
82026-10-27T05:26:09.952ZINFOslog-rs (mgd): interface monitor started module = unnumbered manager unit = daemon
92026-10-27T05:26:09.952ZINFOslog-rs (bgp): dispatcher started module = neighbor unit = dispatcher
102026-10-27T05:26:09.952ZINFOslog-rs (bgp): starting listener with bind arg: [::]:179 module = neighbor unit = dispatcher
112026-10-27T05:26:09.952ZINFOslog-rs (bgp): TcpListener created listener = TcpListener { addr: [::]:179, fd: 10 } module = neighbor unit = dispatcher
122026-10-27T05:26:09.952ZINFOslog-rs (bgp): transitioning to accept loop bind_addr = [::]:179 module = neighbor unit = dispatcher
132026-10-27T05:26:09.952ZINFOslog-rs (mgd): signal handler got context module = admin unit = signal
142026-10-27T05:26:09.955ZDEBGslog-rs: client request body = None method = GET unit = gateway-client uri = http://[::1]:12225/local/switch-id
152026-10-27T05:26:09.956ZDEBGslog-rs: client response result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [::1]:12225, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) }) unit = gateway-client
162026-10-27T05:26:09.956ZERROslog-rs: failed to resolve switch slot error = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id)
172026-10-27T05:26:09.956ZINFOslog-rs (mgd): starting bgp routers: {} module = admin unit = daemon
182026-10-27T05:26:09.956ZINFOslog-rs (mgd): starting bfd sessions: [] module = admin unit = daemon
192026-10-27T05:26:09.956ZERROslog-rs (mgd): failed to start stats server: underlay address not found error = underlay address not found module = admin params = hostname oxz_switch, rack e0cc3c66-bd36-41e3-ac86-ac021dbfa31c, sled 842ea4d7-f783-480a-9d3f-9298e024f004 unit = daemon
202026-10-27T05:26:09.957ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
212026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bestpath/config/fanout unit = api_server versions = all ending with 2.0.0
222026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bestpath/config/fanout unit = api_server versions = all ending with 2.0.0
232026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bfd/peers unit = api_server versions = all starting from 12.0.0
242026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bfd/peers unit = api_server versions = from 1.0.0 to 12.0.0
252026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bfd/peers unit = api_server versions = all starting from 12.0.0
262026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bfd/peers unit = api_server versions = from 1.0.0 to 12.0.0
272026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /bfd/peers/{addr} unit = api_server versions = all
282026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/clear/neighbor unit = api_server versions = all starting from 4.0.0
292026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/clear/neighbor unit = api_server versions = all ending with 4.0.0
302026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/clear/unnumbered-neighbor unit = api_server versions = all starting from 5.0.0
312026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /bgp/config/checker unit = api_server versions = all
322026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/checker unit = api_server versions = all
332026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/checker unit = api_server versions = all
342026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/checker unit = api_server versions = all
352026-10-27T05:26:09.958ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /bgp/config/neighbor unit = api_server versions = all ending with 5.0.0
362026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/neighbor unit = api_server versions = from 4.0.0 to 5.0.0
372026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/neighbor unit = api_server versions = all ending with 4.0.0
382026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/neighbor unit = api_server versions = all starting from 11.0.0
392026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/neighbor unit = api_server versions = from 8.0.0 to 11.0.0
402026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/neighbor unit = api_server versions = from 4.0.0 to 8.0.0
412026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/neighbor unit = api_server versions = all ending with 4.0.0
422026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/neighbor unit = api_server versions = all starting from 11.0.0
432026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/neighbor unit = api_server versions = from 8.0.0 to 11.0.0
442026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/neighbor unit = api_server versions = from 4.0.0 to 8.0.0
452026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/neighbor unit = api_server versions = all ending with 4.0.0
462026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /bgp/config/neighbor/{asn}/{peer} unit = api_server versions = all starting from 5.0.0
472026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/neighbor/{asn}/{peer} unit = api_server versions = all starting from 11.0.0
482026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/neighbor/{asn}/{peer} unit = api_server versions = from 8.0.0 to 11.0.0
492026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/neighbor/{asn}/{peer} unit = api_server versions = from 5.0.0 to 8.0.0
502026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/neighbors unit = api_server versions = from 4.0.0 to 5.0.0
512026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/neighbors unit = api_server versions = all ending with 4.0.0
522026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/neighbors/{asn} unit = api_server versions = all starting from 11.0.0
532026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/neighbors/{asn} unit = api_server versions = from 8.0.0 to 11.0.0
542026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/neighbors/{asn} unit = api_server versions = from 5.0.0 to 8.0.0
552026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /bgp/config/origin4 unit = api_server versions = all
562026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/origin4 unit = api_server versions = all starting from 11.0.0
572026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/origin4 unit = api_server versions = all ending with 11.0.0
582026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/origin4 unit = api_server versions = all starting from 11.0.0
592026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/origin4 unit = api_server versions = all ending with 11.0.0
602026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/origin4 unit = api_server versions = all starting from 11.0.0
612026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/origin4 unit = api_server versions = all ending with 11.0.0
622026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /bgp/config/origin6 unit = api_server versions = all starting from 2.0.0
632026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/origin6 unit = api_server versions = all starting from 11.0.0
642026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/origin6 unit = api_server versions = from 2.0.0 to 11.0.0
652026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/origin6 unit = api_server versions = all starting from 11.0.0
662026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/origin6 unit = api_server versions = from 2.0.0 to 11.0.0
672026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/origin6 unit = api_server versions = all starting from 11.0.0
682026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/origin6 unit = api_server versions = from 2.0.0 to 11.0.0
692026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /bgp/config/router unit = api_server versions = all
702026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/router unit = api_server versions = all
712026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/router unit = api_server versions = all
722026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/router unit = api_server versions = all
732026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/routers unit = api_server versions = all
742026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /bgp/config/shaper unit = api_server versions = all
752026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/shaper unit = api_server versions = all
762026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/shaper unit = api_server versions = all
772026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/shaper unit = api_server versions = all
782026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /bgp/config/unnumbered-neighbor unit = api_server versions = all starting from 5.0.0
792026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/unnumbered-neighbor unit = api_server versions = all starting from 11.0.0
802026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/unnumbered-neighbor unit = api_server versions = from 8.0.0 to 11.0.0
812026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/unnumbered-neighbor unit = api_server versions = from 5.0.0 to 8.0.0
822026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/unnumbered-neighbor unit = api_server versions = all starting from 11.0.0
832026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/unnumbered-neighbor unit = api_server versions = from 8.0.0 to 11.0.0
842026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/config/unnumbered-neighbor unit = api_server versions = from 5.0.0 to 8.0.0
852026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/unnumbered-neighbor unit = api_server versions = all starting from 11.0.0
862026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/unnumbered-neighbor unit = api_server versions = from 8.0.0 to 11.0.0
872026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /bgp/config/unnumbered-neighbor unit = api_server versions = from 5.0.0 to 8.0.0
882026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/unnumbered-neighbors unit = api_server versions = all starting from 11.0.0
892026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/unnumbered-neighbors unit = api_server versions = from 8.0.0 to 11.0.0
902026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/config/unnumbered-neighbors unit = api_server versions = from 5.0.0 to 8.0.0
912026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/history/fsm unit = api_server versions = all starting from 5.0.0
922026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/history/fsm unit = api_server versions = from 2.0.0 to 5.0.0
932026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/history/message unit = api_server versions = all starting from 11.0.0
942026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/history/message unit = api_server versions = from 5.0.0 to 11.0.0
952026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/history/message unit = api_server versions = from 4.0.0 to 5.0.0
962026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/history/message unit = api_server versions = from 2.0.0 to 4.0.0
972026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/message-history unit = api_server versions = all ending with 2.0.0
982026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/omicron/apply unit = api_server versions = all starting from 11.0.0
992026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/omicron/apply unit = api_server versions = from 8.0.0 to 11.0.0
1002026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/omicron/apply unit = api_server versions = from 4.0.0 to 8.0.0
1012026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /bgp/omicron/apply unit = api_server versions = all ending with 4.0.0
1022026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/exported unit = api_server versions = all starting from 11.0.0
1032026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/exported unit = api_server versions = from 6.0.0 to 11.0.0
1042026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/exported unit = api_server versions = from 5.0.0 to 6.0.0
1052026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/exported unit = api_server versions = all ending with 5.0.0
1062026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/imported unit = api_server versions = all ending with 2.0.0
1072026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/neighbors unit = api_server versions = all starting from 11.0.0
1082026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/neighbors unit = api_server versions = from 5.0.0 to 11.0.0
1092026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/neighbors unit = api_server versions = from 4.0.0 to 5.0.0
1102026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/neighbors unit = api_server versions = from 2.0.0 to 4.0.0
1112026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/neighbors unit = api_server versions = all ending with 2.0.0
1122026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /bgp/status/selected unit = api_server versions = all ending with 2.0.0
1132026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /ndp/interface unit = api_server versions = all starting from 5.0.0
1142026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /ndp/interfaces unit = api_server versions = all starting from 5.0.0
1152026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /ndp/manager unit = api_server versions = all starting from 5.0.0
1162026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /rib/config/bestpath/fanout unit = api_server versions = all starting from 2.0.0
1172026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = POST module = admin path = /rib/config/bestpath/fanout unit = api_server versions = all starting from 2.0.0
1182026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /rib/status/imported unit = api_server versions = all starting from 5.0.0
1192026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /rib/status/imported unit = api_server versions = from 2.0.0 to 5.0.0
1202026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /rib/status/selected unit = api_server versions = all starting from 5.0.0
1212026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /rib/status/selected unit = api_server versions = from 2.0.0 to 5.0.0
1222026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /static/route4 unit = api_server versions = all starting from 11.0.0
1232026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /static/route4 unit = api_server versions = from 10.0.0 to 11.0.0
1242026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /static/route4 unit = api_server versions = all ending with 10.0.0
1252026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /static/route4 unit = api_server versions = all starting from 5.0.0
1262026-10-27T05:26:09.959ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /static/route4 unit = api_server versions = all ending with 5.0.0
1272026-10-27T05:26:09.960ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /static/route4 unit = api_server versions = all starting from 11.0.0
1282026-10-27T05:26:09.960ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /static/route4 unit = api_server versions = from 10.0.0 to 11.0.0
1292026-10-27T05:26:09.960ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /static/route4 unit = api_server versions = all ending with 10.0.0
1302026-10-27T05:26:09.960ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /static/route6 unit = api_server versions = all starting from 11.0.0
1312026-10-27T05:26:09.960ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = DELETE module = admin path = /static/route6 unit = api_server versions = from 2.0.0 to 11.0.0
1322026-10-27T05:26:09.960ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /static/route6 unit = api_server versions = all starting from 5.0.0
1332026-10-27T05:26:09.960ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /static/route6 unit = api_server versions = from 2.0.0 to 5.0.0
1342026-10-27T05:26:09.960ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /static/route6 unit = api_server versions = all starting from 11.0.0
1352026-10-27T05:26:09.960ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = PUT module = admin path = /static/route6 unit = api_server versions = from 2.0.0 to 11.0.0
1362026-10-27T05:26:09.960ZDEBGslog-rs (mgd): registered endpoint local_addr = [::]:4676 method = GET module = admin path = /switch/identifiers unit = api_server versions = all starting from 3.0.0
1372026-10-27T05:26:09.960ZINFOslog-rs (mgd): listening local_addr = [::]:4676 module = admin unit = api_server
1382026-10-27T05:26:09.960ZDEBGslog-rs (mgd): successfully registered DTrace USDT probes local_addr = [::]:4676 module = admin unit = api_server
1392026-10-27T05:26:09.960ZINFOslog-rs: listening on [::]:4676
1402026-10-27T05:26:10.957ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
1412026-10-27T05:26:11.958ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
1422026-10-27T05:26:12.958ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
1432026-10-27T05:26:13.959ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
1442026-10-27T05:26:14.959ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
1452026-10-27T05:26:15.973ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
1462026-10-27T05:26:16.973ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
1472026-10-27T05:26:17.974ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
1482026-10-27T05:26:18.974ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
1492026-10-27T05:26:19.957ZDEBGslog-rs: client request body = None method = GET unit = gateway-client uri = http://[::1]:12225/local/switch-id
1502026-10-27T05:26:19.957ZDEBGslog-rs: client response result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [::1]:12225, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) }) unit = gateway-client
1512026-10-27T05:26:19.957ZERROslog-rs: failed to resolve switch slot error = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id)
1522026-10-27T05:26:19.974ZWARNslog-rs (mg-lower): failed to ensure TEP address fd04:c61:cf13:c40f::1 on ASIC: Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) error = Communication Error: error sending request for url (http://localhost:12224/loopback/ipv6) module = sync prefix = fd04:c61:cf13:c40f::1 unit = dpd
153 [ Oct 27 05:26:21 Rereading configuration. ]
154 [ Oct 27 05:26:21 Executing refresh method ("/opt/oxide/mgd/pkg/refresh_ctrun_child.sh -c 213"). ]
155 + CONTRACT=''
156 + getopts c: o
157 + CONTRACT=213
158 + getopts c: o
159 + [[ -n 213 ]]
160 + pgrep -c 213
161 + PARENT_PID=2299
162 + [[ -n 2299 ]]
163 + pgrep -P 2299
164 + CHILD_PID=2300
165 + [[ -n 2300 ]]
166 + kill -USR1 2300
1672026-10-27T05:26:21.037ZINFOslog-rs (mgd): handling smf refresh module = admin unit = smf
1682026-10-27T05:26:21.039ZINFOslog-rs (mgd): starting stats server on smf refresh module = admin unit = smf
1692026-10-27T05:26:21.039ZINFOslog-rs (mgd): started stats server on smf refresh module = admin unit = smf
170 [ Oct 27 05:26:21 Method "refresh" exited with status 0. ]
171 Oct 27 05:26:21.040 DEBG registered DTrace probes
172 Oct 27 05:26:21.040 DEBG registered endpoint, versions: all, path: /{producer_id}, method: GET, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
173 Oct 27 05:26:21.040 INFO listening, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:335
174 Oct 27 05:26:21.040 DEBG successfully registered DTrace USDT probes, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot
175 Oct 27 05:26:21.041 DEBG Nexus IP not provided, will use DNS to resolve it
176 Oct 27 05:26:21.041 INFO new DNS resolver, addresses: [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53], component: internal-dns-resolver, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/internal-dns/resolver/src/resolver.rs:144
177 Oct 27 05:26:21.041 DEBG starting producer registration task
178 Oct 27 05:26:21.041 INFO starting oximeter metric producer server, interval: 1s, address: [fd00:1122:3344:101::2]:4677, producer_id: d8fc593d-c9d6-4899-9580-a62adf75b991, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:280
179 Oct 27 05:26:21.041 DEBG registering as oximeter producer with Nexus, registration_count: 0, component: producer-registration-task
180 Oct 27 05:26:21.042 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2e6d4947-2419-4229-9c7c-941308f8fe1c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8790fd52-de05-48b0-9d59-ab53c81a53e2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 5370, tv_nsec: 86766550 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
181 Oct 27 05:26:21.043 DEBG will register with Nexus at [fd00:1122:3344:101::c]:12221, component: producer-registration-task
182 Oct 27 05:26:21.044 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::c]:12221/metrics/producers, method: POST, component: producer-registration-task
1832026-10-27T05:26:25.733ZINFOslog-rs (mgd): accepted connection local_addr = [::]:4676 module = admin remote_addr = [fd00:1122:3344:101::1]:63287 unit = api_server
1842026-10-27T05:26:25.733ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:63287 req_id = eaf82720-e326-4206-b235-d6485fa7528b unit = api_server uri = /static/route4 version = 12.0.0
1852026-10-27T05:26:25.734ZINFOslog-rs (mgd): request completed latency_us = 238 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:63287 req_id = eaf82720-e326-4206-b235-d6485fa7528b response_code = 200 unit = api_server uri = /static/route4
1862026-10-27T05:26:25.734ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:63287 req_id = ed19800e-a155-40e0-967d-de64dbe2fff5 unit = api_server uri = /static/route6 version = 12.0.0
1872026-10-27T05:26:25.734ZINFOslog-rs (mgd): request completed latency_us = 67 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:63287 req_id = ed19800e-a155-40e0-967d-de64dbe2fff5 response_code = 200 unit = api_server uri = /static/route6
1882026-10-27T05:26:25.734ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = PUT module = admin remote_addr = [fd00:1122:3344:101::1]:63287 req_id = d7b9287d-2a64-49a9-930e-19e524e0fa8f unit = api_server uri = /static/route4 version = 12.0.0
1892026-10-27T05:26:25.735ZINFOslog-rs (mgd): request completed latency_us = 596 local_addr = [::]:4676 method = PUT module = admin remote_addr = [fd00:1122:3344:101::1]:63287 req_id = d7b9287d-2a64-49a9-930e-19e524e0fa8f response_code = 204 unit = api_server uri = /static/route4
1902026-10-27T05:26:25.735ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:63287 req_id = db7ebbf9-149d-479f-b328-828de614325b unit = api_server uri = /bgp/config/routers version = 12.0.0
1912026-10-27T05:26:25.735ZINFOslog-rs (mgd): request completed latency_us = 107 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:63287 req_id = db7ebbf9-149d-479f-b328-828de614325b response_code = 200 unit = api_server uri = /bgp/config/routers
1922026-10-27T05:26:25.735ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:63287 req_id = c4e011ab-afe3-47a0-8d39-f4248f6b8f27 unit = api_server uri = /bfd/peers version = 12.0.0
1932026-10-27T05:26:25.735ZINFOslog-rs (mgd): request completed latency_us = 105 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:63287 req_id = c4e011ab-afe3-47a0-8d39-f4248f6b8f27 response_code = 200 unit = api_server uri = /bfd/peers
1942026-10-27T05:26:25.736ZERROslog-rs (mg-lower): handling change failed: route lookup error io error No such process (os error 3) error = route lookup error io error No such process (os error 3) module = sync unit = event_loop
1952026-10-27T05:26:25.736ZINFOslog-rs (mg-lower): restarting sync loop module = sync unit = event_loop
1962026-10-27T05:26:29.958ZDEBGslog-rs: client request body = None method = GET unit = gateway-client uri = http://[::1]:12225/local/switch-id
1972026-10-27T05:26:29.958ZDEBGslog-rs: client response result = Ok(Response { url: "http://[::1]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f5de8f5-7056-43b9-9acb-50d071a4c667", "content-length": "26", "date": "Tue, 27 Oct 2026 05:26:29 GMT"} }) unit = gateway-client
1982026-10-27T05:26:29.958ZINFOslog-rs: we are in switch slot 0
199 Oct 27 05:26:36.391 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
200 Oct 27 05:26:36.391 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out, delay: 15.004546438s, count: 0, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
201 Oct 27 05:26:36.614 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2e6d4947-2419-4229-9c7c-941308f8fe1c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8790fd52-de05-48b0-9d59-ab53c81a53e2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 5385, tv_nsec: 311909150 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
202 Oct 27 05:26:36.615 DEBG will register with Nexus at [fd00:1122:3344:101::c]:12221, component: producer-registration-task
203 Oct 27 05:26:36.616 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::c]:12221/metrics/producers, method: POST, component: producer-registration-task
204 Oct 27 05:26:51.655 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
205 Oct 27 05:26:51.655 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out, delay: 30.229082973s, count: 1, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
206 Oct 27 05:26:52.267 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2e6d4947-2419-4229-9c7c-941308f8fe1c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8790fd52-de05-48b0-9d59-ab53c81a53e2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 5400, tv_nsec: 926762298 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
207 Oct 27 05:26:52.267 DEBG will register with Nexus at [fd00:1122:3344:101::c]:12221, component: producer-registration-task
208 Oct 27 05:26:52.269 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::c]:12221/metrics/producers, method: POST, component: producer-registration-task
2092026-10-27T05:26:56.120ZINFOslog-rs (mgd): accepted connection local_addr = [::]:4676 module = admin remote_addr = [fd00:1122:3344:101::1]:45035 unit = api_server
2102026-10-27T05:26:56.184ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:45035 req_id = 40d991fd-9daa-42c2-9cb0-869a55d839e9 unit = api_server uri = /static/route4 version = 12.0.0
2112026-10-27T05:26:56.184ZINFOslog-rs (mgd): request completed latency_us = 1135 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:45035 req_id = 40d991fd-9daa-42c2-9cb0-869a55d839e9 response_code = 200 unit = api_server uri = /static/route4
2122026-10-27T05:26:56.184ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:45035 req_id = 598d92c4-6f84-4bd4-8b0a-cc089205a632 unit = api_server uri = /static/route6 version = 12.0.0
2132026-10-27T05:26:56.184ZINFOslog-rs (mgd): request completed latency_us = 38 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:45035 req_id = 598d92c4-6f84-4bd4-8b0a-cc089205a632 response_code = 200 unit = api_server uri = /static/route6
2142026-10-27T05:26:56.184ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:45035 req_id = 62c4fcf7-7c56-4e1c-8308-046c4b3be9b2 unit = api_server uri = /bgp/config/routers version = 12.0.0
2152026-10-27T05:26:56.184ZINFOslog-rs (mgd): request completed latency_us = 32 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:45035 req_id = 62c4fcf7-7c56-4e1c-8308-046c4b3be9b2 response_code = 200 unit = api_server uri = /bgp/config/routers
2162026-10-27T05:26:56.184ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:45035 req_id = a29fc651-55b4-4b69-82ca-478d31afb83e unit = api_server uri = /bfd/peers version = 12.0.0
2172026-10-27T05:26:56.184ZINFOslog-rs (mgd): request completed latency_us = 80 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:45035 req_id = a29fc651-55b4-4b69-82ca-478d31afb83e response_code = 200 unit = api_server uri = /bfd/peers
218 Oct 27 05:27:07.268 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
219 Oct 27 05:27:07.268 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out, delay: 45.843747179s, count: 2, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
220 Oct 27 05:27:08.042 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2e6d4947-2419-4229-9c7c-941308f8fe1c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8790fd52-de05-48b0-9d59-ab53c81a53e2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 5416, tv_nsec: 702752732 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
221 Oct 27 05:27:08.043 DEBG will register with Nexus at [fd00:1122:3344:101::c]:12221, component: producer-registration-task
222 Oct 27 05:27:08.043 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::c]:12221/metrics/producers, method: POST, component: producer-registration-task
223 Oct 27 05:27:23.542 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
224 Oct 27 05:27:23.794 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out, delay: 62.338778917s, count: 3, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
225 Oct 27 05:27:26.132 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2e6d4947-2419-4229-9c7c-941308f8fe1c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8790fd52-de05-48b0-9d59-ab53c81a53e2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 5434, tv_nsec: 789295864 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
226 Oct 27 05:27:26.140 DEBG will register with Nexus at [fd00:1122:3344:101::c]:12221, component: producer-registration-task
227 Oct 27 05:27:26.163 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::c]:12221/metrics/producers, method: POST, component: producer-registration-task
2282026-10-27T05:27:26.190ZINFOslog-rs (mgd): accepted connection local_addr = [::]:4676 module = admin remote_addr = [fd00:1122:3344:101::1]:57474 unit = api_server
2292026-10-27T05:27:26.192ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:57474 req_id = df68f7ab-c16e-4712-b8c8-9b84b5018693 unit = api_server uri = /static/route4 version = 12.0.0
2302026-10-27T05:27:26.220ZINFOslog-rs (mgd): request completed latency_us = 29789 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:57474 req_id = df68f7ab-c16e-4712-b8c8-9b84b5018693 response_code = 200 unit = api_server uri = /static/route4
2312026-10-27T05:27:26.224ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:57474 req_id = 1a9cb156-4b0c-4046-a9df-3e39b46b794c unit = api_server uri = /static/route6 version = 12.0.0
2322026-10-27T05:27:26.224ZINFOslog-rs (mgd): request completed latency_us = 158 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:57474 req_id = 1a9cb156-4b0c-4046-a9df-3e39b46b794c response_code = 200 unit = api_server uri = /static/route6
2332026-10-27T05:27:26.226ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:57474 req_id = 700a2c74-4672-4d75-9048-54c5796ee3ba unit = api_server uri = /bgp/config/routers version = 12.0.0
2342026-10-27T05:27:26.227ZINFOslog-rs (mgd): request completed latency_us = 1750 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:57474 req_id = 700a2c74-4672-4d75-9048-54c5796ee3ba response_code = 200 unit = api_server uri = /bgp/config/routers
2352026-10-27T05:27:26.237ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:57474 req_id = 4ea48476-dc07-45ff-9e75-455616cc2c5a unit = api_server uri = /bfd/peers version = 12.0.0
2362026-10-27T05:27:26.245ZINFOslog-rs (mgd): request completed latency_us = 8878 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:57474 req_id = 4ea48476-dc07-45ff-9e75-455616cc2c5a response_code = 200 unit = api_server uri = /bfd/peers
237 Oct 27 05:27:41.163 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: TimedOut }), component: producer-registration-task
238 Oct 27 05:27:41.165 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out, delay: 79.740789816s, count: 4, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
239 Oct 27 05:27:43.832 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2e6d4947-2419-4229-9c7c-941308f8fe1c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8790fd52-de05-48b0-9d59-ab53c81a53e2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 5452, tv_nsec: 493139455 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
240 Oct 27 05:27:43.833 DEBG will register with Nexus at [fd00:1122:3344:101::c]:12221, component: producer-registration-task
241 Oct 27 05:27:43.835 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::c]:12221/metrics/producers, method: POST, component: producer-registration-task
242 Oct 27 05:27:43.837 DEBG client response, result: Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::c]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) }), component: producer-registration-task
243 Oct 27 05:27:43.838 WARN failed to register with Nexus, will retry, error: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): client error (Connect): tcp connect error: Connection refused (os error 146), delay: 82.412925457s, count: 5, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
244 Oct 27 05:27:53.006 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2e6d4947-2419-4229-9c7c-941308f8fe1c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8790fd52-de05-48b0-9d59-ab53c81a53e2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 5461, tv_nsec: 666411215 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
245 Oct 27 05:27:53.006 DEBG will register with Nexus at [fd00:1122:3344:101::c]:12221, component: producer-registration-task
246 Oct 27 05:27:53.007 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::c]:12221/metrics/producers, method: POST, component: producer-registration-task
247 Oct 27 05:27:53.071 DEBG client response, result: Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "75fe5a46-d8f9-4948-bcd8-1a02ed90a221", "content-length": "133", "date": "Tue, 27 Oct 2026 05:27:53 GMT"} }), component: producer-registration-task
248 Oct 27 05:27:53.072 WARN failed to register with Nexus, will retry, error: Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "75fe5a46-d8f9-4948-bcd8-1a02ed90a221", "content-length": "133", "date": "Tue, 27 Oct 2026 05:27:53 GMT"}; value: Error { error_code: Some("ServiceNotAvailable"), message: "Service Unavailable", request_id: "75fe5a46-d8f9-4948-bcd8-1a02ed90a221" }, delay: 91.647664438s, count: 6, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
2492026-10-27T05:27:56.247ZINFOslog-rs (mgd): accepted connection local_addr = [::]:4676 module = admin remote_addr = [fd00:1122:3344:101::1]:43645 unit = api_server
2502026-10-27T05:27:56.247ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:43645 req_id = c4a02d29-f633-4612-8147-555deece3b29 unit = api_server uri = /static/route4 version = 12.0.0
2512026-10-27T05:27:56.247ZINFOslog-rs (mgd): request completed latency_us = 276 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:43645 req_id = c4a02d29-f633-4612-8147-555deece3b29 response_code = 200 unit = api_server uri = /static/route4
2522026-10-27T05:27:56.247ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:43645 req_id = db6b6754-ae25-4d14-aebc-5201ebd9c0d7 unit = api_server uri = /static/route6 version = 12.0.0
2532026-10-27T05:27:56.248ZINFOslog-rs (mgd): request completed latency_us = 85 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:43645 req_id = db6b6754-ae25-4d14-aebc-5201ebd9c0d7 response_code = 200 unit = api_server uri = /static/route6
2542026-10-27T05:27:56.248ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:43645 req_id = 178c238f-f824-4e6a-a448-fe6bee2bd31c unit = api_server uri = /bgp/config/routers version = 12.0.0
2552026-10-27T05:27:56.248ZINFOslog-rs (mgd): request completed latency_us = 70 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:43645 req_id = 178c238f-f824-4e6a-a448-fe6bee2bd31c response_code = 200 unit = api_server uri = /bgp/config/routers
2562026-10-27T05:27:56.248ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:43645 req_id = 3847e57b-8ae1-4f75-97e5-3bc5998dde97 unit = api_server uri = /bfd/peers version = 12.0.0
2572026-10-27T05:27:56.248ZINFOslog-rs (mgd): request completed latency_us = 47 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:43645 req_id = 3847e57b-8ae1-4f75-97e5-3bc5998dde97 response_code = 200 unit = api_server uri = /bfd/peers
258 Oct 27 05:28:01.272 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2e6d4947-2419-4229-9c7c-941308f8fe1c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8790fd52-de05-48b0-9d59-ab53c81a53e2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 5469, tv_nsec: 933027839 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
259 Oct 27 05:28:01.273 DEBG will register with Nexus at [fd00:1122:3344:101::c]:12221, component: producer-registration-task
260 Oct 27 05:28:01.274 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::c]:12221/metrics/producers, method: POST, component: producer-registration-task
261 Oct 27 05:28:01.280 DEBG client response, result: Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "2b9c0342-8838-44d3-8958-c52391b6c490", "content-length": "133", "date": "Tue, 27 Oct 2026 05:28:01 GMT"} }), component: producer-registration-task
262 Oct 27 05:28:01.280 WARN failed to register with Nexus, will retry, error: Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "2b9c0342-8838-44d3-8958-c52391b6c490", "content-length": "133", "date": "Tue, 27 Oct 2026 05:28:01 GMT"}; value: Error { error_code: Some("ServiceNotAvailable"), message: "Service Unavailable", request_id: "2b9c0342-8838-44d3-8958-c52391b6c490" }, delay: 99.855295246s, count: 7, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
2632026-10-27T05:28:26.249ZINFOslog-rs (mgd): accepted connection local_addr = [::]:4676 module = admin remote_addr = [fd00:1122:3344:101::1]:48261 unit = api_server
2642026-10-27T05:28:26.249ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:48261 req_id = 3f4a8d11-198c-41a3-b92d-707812179c3f unit = api_server uri = /static/route4 version = 12.0.0
2652026-10-27T05:28:26.249ZINFOslog-rs (mgd): request completed latency_us = 95 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:48261 req_id = 3f4a8d11-198c-41a3-b92d-707812179c3f response_code = 200 unit = api_server uri = /static/route4
2662026-10-27T05:28:26.249ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:48261 req_id = e0d4f211-3b84-4755-99eb-4adf8543fc5d unit = api_server uri = /static/route6 version = 12.0.0
2672026-10-27T05:28:26.249ZINFOslog-rs (mgd): request completed latency_us = 35 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:48261 req_id = e0d4f211-3b84-4755-99eb-4adf8543fc5d response_code = 200 unit = api_server uri = /static/route6
2682026-10-27T05:28:26.249ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:48261 req_id = abeb3375-bf66-4dc8-ad99-52652fdbf94e unit = api_server uri = /bgp/config/routers version = 12.0.0
2692026-10-27T05:28:26.249ZINFOslog-rs (mgd): request completed latency_us = 40 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:48261 req_id = abeb3375-bf66-4dc8-ad99-52652fdbf94e response_code = 200 unit = api_server uri = /bgp/config/routers
2702026-10-27T05:28:26.250ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:48261 req_id = 20cae896-8fe6-4c04-95ea-1146077739ce unit = api_server uri = /bfd/peers version = 12.0.0
2712026-10-27T05:28:26.250ZINFOslog-rs (mgd): request completed latency_us = 26 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:48261 req_id = 20cae896-8fe6-4c04-95ea-1146077739ce response_code = 200 unit = api_server uri = /bfd/peers
272 Oct 27 05:28:32.023 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2e6d4947-2419-4229-9c7c-941308f8fe1c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8790fd52-de05-48b0-9d59-ab53c81a53e2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 5500, tv_nsec: 684465988 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
273 Oct 27 05:28:32.024 DEBG will register with Nexus at [fd00:1122:3344:101::c]:12221, component: producer-registration-task
274 Oct 27 05:28:32.025 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::c]:12221/metrics/producers, method: POST, component: producer-registration-task
275 Oct 27 05:28:32.030 DEBG client response, result: Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "4e9e406c-04e6-4713-8c6f-885578c8a017", "content-length": "133", "date": "Tue, 27 Oct 2026 05:28:32 GMT"} }), component: producer-registration-task
276 Oct 27 05:28:32.030 WARN failed to register with Nexus, will retry, error: Error Response: status: 503 Service Unavailable; headers: {"content-type": "application/json", "x-request-id": "4e9e406c-04e6-4713-8c6f-885578c8a017", "content-length": "133", "date": "Tue, 27 Oct 2026 05:28:32 GMT"}; value: Error { error_code: Some("ServiceNotAvailable"), message: "Service Unavailable", request_id: "4e9e406c-04e6-4713-8c6f-885578c8a017" }, delay: 130.605904027s, count: 8, component: producer-registration-task, file: /home/build/.cargo/git/checkouts/omicron-e0115f488d1c1d58/d6fba55/oximeter/producer/src/lib.rs:376
2772026-10-27T05:28:50.984ZINFOslog-rs (mgd): accepted connection local_addr = [::]:4676 module = admin remote_addr = [fd00:1122:3344:101::1]:37908 unit = api_server
2782026-10-27T05:28:50.984ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:37908 req_id = deb8c317-5436-4370-aa84-23c0d94d2716 unit = api_server uri = /static/route4 version = 12.0.0
2792026-10-27T05:28:50.984ZINFOslog-rs (mgd): request completed latency_us = 394 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:37908 req_id = deb8c317-5436-4370-aa84-23c0d94d2716 response_code = 200 unit = api_server uri = /static/route4
2802026-10-27T05:28:50.985ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:37908 req_id = a46eb138-ae39-440c-8c55-f40d85b9f253 unit = api_server uri = /static/route6 version = 12.0.0
2812026-10-27T05:28:50.985ZINFOslog-rs (mgd): request completed latency_us = 83 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:37908 req_id = a46eb138-ae39-440c-8c55-f40d85b9f253 response_code = 200 unit = api_server uri = /static/route6
2822026-10-27T05:28:50.985ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:37908 req_id = 890ad480-e26d-45a6-a259-3f45608b1964 unit = api_server uri = /bgp/config/routers version = 12.0.0
2832026-10-27T05:28:50.985ZINFOslog-rs (mgd): request completed latency_us = 49 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:37908 req_id = 890ad480-e26d-45a6-a259-3f45608b1964 response_code = 200 unit = api_server uri = /bgp/config/routers
2842026-10-27T05:28:50.985ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:37908 req_id = 20703a20-d654-46f1-aca9-0fec57b72912 unit = api_server uri = /bfd/peers version = 12.0.0
2852026-10-27T05:28:50.985ZINFOslog-rs (mgd): request completed latency_us = 39 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:37908 req_id = 20703a20-d654-46f1-aca9-0fec57b72912 response_code = 200 unit = api_server uri = /bfd/peers
2862026-10-27T05:29:20.986ZINFOslog-rs (mgd): accepted connection local_addr = [::]:4676 module = admin remote_addr = [fd00:1122:3344:101::1]:58238 unit = api_server
2872026-10-27T05:29:20.987ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:58238 req_id = dd77a70d-6930-4775-bd07-c854bf178200 unit = api_server uri = /static/route4 version = 12.0.0
2882026-10-27T05:29:20.987ZINFOslog-rs (mgd): request completed latency_us = 88 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:58238 req_id = dd77a70d-6930-4775-bd07-c854bf178200 response_code = 200 unit = api_server uri = /static/route4
2892026-10-27T05:29:20.987ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:58238 req_id = cf9fab28-02b8-4a18-a805-b8c6e9a547ef unit = api_server uri = /static/route6 version = 12.0.0
2902026-10-27T05:29:20.987ZINFOslog-rs (mgd): request completed latency_us = 81 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:58238 req_id = cf9fab28-02b8-4a18-a805-b8c6e9a547ef response_code = 200 unit = api_server uri = /static/route6
2912026-10-27T05:29:20.987ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:58238 req_id = 06b40dc8-8596-4395-8d99-2778f4d9ff9d unit = api_server uri = /bgp/config/routers version = 12.0.0
2922026-10-27T05:29:20.987ZINFOslog-rs (mgd): request completed latency_us = 43 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:58238 req_id = 06b40dc8-8596-4395-8d99-2778f4d9ff9d response_code = 200 unit = api_server uri = /bgp/config/routers
2932026-10-27T05:29:20.987ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:58238 req_id = 3fea55f7-0679-4c8d-8e38-a2ba49ddac40 unit = api_server uri = /bfd/peers version = 12.0.0
2942026-10-27T05:29:20.988ZINFOslog-rs (mgd): request completed latency_us = 26 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:58238 req_id = 3fea55f7-0679-4c8d-8e38-a2ba49ddac40 response_code = 200 unit = api_server uri = /bfd/peers
295 Oct 27 05:29:42.429 DEBG lookup_socket_v6 srv, response: SrvLookup(Lookup { query: Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("2e6d4947-2419-4229-9c7c-941308f8fe1c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("8790fd52-de05-48b0-9d59-ab53c81a53e2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nexus._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12221, target: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c20a5c79-32cb-4f44-8206-f070863d9653.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::b)) }], valid_until: Instant { tv_sec: 5571, tv_nsec: 90543893 } }), dns_name: _nexus._tcp.control-plane.oxide.internal, component: internal-dns-resolver
296 Oct 27 05:29:42.429 DEBG will register with Nexus at [fd00:1122:3344:101::c]:12221, component: producer-registration-task
297 Oct 27 05:29:42.430 DEBG client request, body: Some(Body), uri: http://[fd00:1122:3344:101::c]:12221/metrics/producers, method: POST, component: producer-registration-task
298 Oct 27 05:29:42.480 DEBG client response, result: Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "a6594dbc-5cb2-41ad-8c19-3f4c8e497ad0", "content-length": "41", "date": "Tue, 27 Oct 2026 05:29:42 GMT"} }), component: producer-registration-task
299 Oct 27 05:29:42.480 DEBG successfully registered with Nexus, pausing until time to renew lease, wait_period: 150s, lease_duration: 600s, component: producer-registration-task
3002026-10-27T05:29:50.989ZINFOslog-rs (mgd): accepted connection local_addr = [::]:4676 module = admin remote_addr = [fd00:1122:3344:101::1]:59049 unit = api_server
3012026-10-27T05:29:50.989ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:59049 req_id = 2fc2217e-fb06-4ee9-a59d-1f8f2879525d unit = api_server uri = /static/route4 version = 12.0.0
3022026-10-27T05:29:50.989ZINFOslog-rs (mgd): request completed latency_us = 70 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:59049 req_id = 2fc2217e-fb06-4ee9-a59d-1f8f2879525d response_code = 200 unit = api_server uri = /static/route4
3032026-10-27T05:29:50.989ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:59049 req_id = 3d15e0af-15d8-4e35-ad18-9819cf2eba0d unit = api_server uri = /static/route6 version = 12.0.0
3042026-10-27T05:29:50.989ZINFOslog-rs (mgd): request completed latency_us = 63 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:59049 req_id = 3d15e0af-15d8-4e35-ad18-9819cf2eba0d response_code = 200 unit = api_server uri = /static/route6
3052026-10-27T05:29:50.989ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:59049 req_id = feb139bb-8fdb-484d-9163-4190fc883638 unit = api_server uri = /bgp/config/routers version = 12.0.0
3062026-10-27T05:29:50.989ZINFOslog-rs (mgd): request completed latency_us = 42 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:59049 req_id = feb139bb-8fdb-484d-9163-4190fc883638 response_code = 200 unit = api_server uri = /bgp/config/routers
3072026-10-27T05:29:50.990ZDEBGslog-rs (mgd): determined request API version local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:59049 req_id = 5163d7d3-03a9-4d16-b37b-0ffa99a1ec22 unit = api_server uri = /bfd/peers version = 12.0.0
3082026-10-27T05:29:50.990ZINFOslog-rs (mgd): request completed latency_us = 41 local_addr = [::]:4676 method = GET module = admin remote_addr = [fd00:1122:3344:101::1]:59049 req_id = 5163d7d3-03a9-4d16-b37b-0ffa99a1ec22 response_code = 200 unit = api_server uri = /bfd/peers
309 Oct 27 05:29:54.862 INFO accepted connection, remote_addr: [fd00:1122:3344:101::d]:45291, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:1057
310 Oct 27 05:29:54.869 INFO request completed, latency_us: 6272, response_code: 200, uri: /d8fc593d-c9d6-4899-9580-a62adf75b991, method: GET, req_id: f3a5a7ed-8a58-4004-a5bc-a75e8b483c44, remote_addr: [fd00:1122:3344:101::d]:45291, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
311 Oct 27 05:29:55.861 INFO request completed, latency_us: 355, response_code: 200, uri: /d8fc593d-c9d6-4899-9580-a62adf75b991, method: GET, req_id: 05dbba16-0d94-4d1d-8cb4-5ade3cd48005, remote_addr: [fd00:1122:3344:101::d]:45291, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
312 Oct 27 05:29:56.861 INFO request completed, latency_us: 494, response_code: 200, uri: /d8fc593d-c9d6-4899-9580-a62adf75b991, method: GET, req_id: 77d206d5-43fb-41de-b9a4-0404f411da36, remote_addr: [fd00:1122:3344:101::d]:45291, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
313 Oct 27 05:29:57.862 INFO request completed, latency_us: 425, response_code: 200, uri: /d8fc593d-c9d6-4899-9580-a62adf75b991, method: GET, req_id: 70f171bc-620d-4399-a0e4-c0c3cefca275, remote_addr: [fd00:1122:3344:101::d]:45291, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
314 Oct 27 05:29:58.862 INFO request completed, latency_us: 394, response_code: 200, uri: /d8fc593d-c9d6-4899-9580-a62adf75b991, method: GET, req_id: a91b3529-9b01-4616-9e97-f77e5286fef9, remote_addr: [fd00:1122:3344:101::d]:45291, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
315 Oct 27 05:29:59.862 INFO request completed, latency_us: 333, response_code: 200, uri: /d8fc593d-c9d6-4899-9580-a62adf75b991, method: GET, req_id: 3409a06b-f699-462b-9f37-8ca1ec5a357a, remote_addr: [fd00:1122:3344:101::d]:45291, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874
316 Oct 27 05:30:00.862 INFO request completed, latency_us: 320, response_code: 200, uri: /d8fc593d-c9d6-4899-9580-a62adf75b991, method: GET, req_id: beac99b0-4ddb-4631-8766-5b070852bcb3, remote_addr: [fd00:1122:3344:101::d]:45291, local_addr: [fd00:1122:3344:101::2]:4677, component: dropshot, file: /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:874